Papers with LaBSE
Improving Multi-lingual Alignment Through Soft Contrastive Learning (2024.naacl-srw)
Copied to clipboard
| Challenge: | Existing methods to train multi-lingual sentence embeddings ruins the mono-lingual space. |
| Approach: | They propose a method to align multi-lingual embeddings based on similarity of sentences measured by a pre-trained mono-lingual teacher model. |
| Outcome: | The proposed method outperforms existing multi-lingual embeddings including LaBSE on five languages and on a translation pair for Tatoeba dataset. |
Robust Fragment-Based Framework for Cross-lingual Sentence Retrieval (2021.findings-emnlp)
Copied to clipboard
Nattapol Trijakwanich, Peerat Limkonchotiwat, Raheem Sarwar, Wannaphong Phatthiyaphaibun, Ekapol Chuangsuwanich, Sarana Nutanong
| Challenge: | Cross-lingual Sentence Retrieval (CLSR) aims at retrieving parallel sentence pairs that are translations of each other from a multilingual set of comparable documents. |
| Approach: | They propose a framework for cross-lingual sentence retrieval that uses a collection of fragments to improve sentence retrievals. |
| Outcome: | The proposed framework improves the retrieval robustness of the base sentences encoded by m-USE, LASER, and LaBSE. |
Multilingual Representation Distillation with Contrastive Learning (2023.eacl-main)
Copied to clipboard
| Challenge: | Contextual representations from large pretrained language models encode semantic information from two or more languages. |
| Approach: | They integrate contrastive learning into multilingual representation distillation and use it for quality estimation of parallel sentences. |
| Outcome: | The proposed model outperforms existing models with similarity searches and filtering tasks across low-resource languages. |
LEALLA: Learning Lightweight Language-agnostic Sentence Embeddings with Knowledge Distillation (2023.eacl-main)
Copied to clipboard
| Challenge: | Large-scale language-agnostic sentence embedding models suffer from inference speed and computation overhead. |
| Approach: | They propose to train a lightweight sentence embedding model to achieve this by incorporating knowledge from a teacher model. |
| Outcome: | The proposed model can build low-dimensional sentences for 109 languages with a thin-deep encoder. |
Probing Cross-Lingual Lexical Knowledge from Multilingual Sentence Encoders (2023.eacl-main)
Copied to clipboard
| Challenge: | Pretrained multilingual language models (LMs) can be 'rewired' into effective multilingual sentence encoders (SEs) however, it remains unclear how to best leverage them to represent sub-sentence lexical items in cross-lingual lexicals. |
| Approach: | They propose a method for exposing cross-lingual lexical knowledge by additional fine-tuning through inexpensive contrastive learning that requires only a small amount of word translation pairs. |
| Outcome: | The proposed method exposes cross-lingual lexical knowledge by additional fine-tuning through inexpensive contrastive learning that requires only a small amount of word translation pairs. |
Are the Best Multilingual Document Embeddings simply Based on Sentence Embeddings? (2023.findings-eacl)
Copied to clipboard
| Challenge: | obtaining document embeddings at document level is challenging due to computational requirements and lack of appropriate data. |
| Approach: | They compare methods to produce document-level representations from sentences based on LASER, LaBSE, and Sentence BERT pre-trained multilingual models. |
| Outcome: | The proposed methods produce document-level representations from sentences in 8 languages . the results show that a clever combination of sentence embeddings is usually better than encoding the full document as a single unit. |
Multilingual Sentence Transformer as A Multilingual Word Aligner (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Multilingual pretrained language models (mPLMs) have shown their effectiveness in multilingual word alignment induction, but these methods usually start from mBERT or XLM-R. |
| Approach: | They propose to fine tune multilingual sentence Transformer LaBSE for alignment induction using parallel corpus and a parallel corpora model. |
| Outcome: | The proposed model outperforms existing models on seven language pairs and achieves new state-of-the-art on zero-shot language pairs. |
Claim Matching Beyond English to Scale Global Fact-Checking (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods to fact-check content are not scaled well in non-English contexts. |
| Approach: | They propose to use a WhatsApp tipline and public group message dataset to find pairs of textual messages containing claims that can be served with one fact-check. |
| Outcome: | The proposed model outperforms existing models in English, Hindi, and Tamil in all settings. |
Critical Learning Periods: Leveraging Early Training Dynamics for Efficient Data Pruning (2024.findings-acl)
Copied to clipboard
| Challenge: | Neural Machine Translation models are extremely data-hungry and require a large dataset to maintain data quality. |
| Approach: | They propose a new data pruning technique that leverages early model training dynamics to identify the most relevant data points for model performance. |
| Outcome: | The proposed technique outperforms the benchmarks on indo-European languages while pruning up to 50% of training data. |
Multilingual and Cross-Lingual Intent Detection from Spoken Data (2021.emnlp-main)
Copied to clipboard
Daniela Gerz, Pei-Hao Su, Razvan Kusztos, Avishek Mondal, Michał Lis, Eshan Singhal, Nikola Mrkšić, Tsung-Hsien Wen, Ivan Vulić
| Challenge: | a systematic study on multilingual and cross-lingual intent detection from spoken data is presented . current work on intent detection is limited to English, and standard benchmarks exist only in English. |
| Approach: | They present a systematic study on multilingual and cross-lingual intent detection from spoken data. |
| Outcome: | The proposed resource is called MInDS-14, and it provides strong intent detection in most target languages. |
BERTifying Sinhala - A Comprehensive Analysis of Pre-trained Language Models for Sinhala Text Classification (2022.lrec-1)
Copied to clipboard
| Challenge: | Large-scale monolingual pre-trained language models have shown promising results for high-resource as well as lowresource languages, especially for text classification. |
| Approach: | They provide a set of recommendations for using pre-trained models for Sinhala text classification and introduce new annotated datasets useful for future research. |
| Outcome: | The proposed models are far superior to existing models for Sinhala and set a strong baseline for text classification when fine-tuned. |
TransAlign: Machine Translation Encoders are Strong Word Aligners, Too (2025.findings-emnlp)
Copied to clipboard
| Challenge: | translation-based approaches to cross-lingual transfer (XLT) are limited. |
| Approach: | They propose a word aligner that utilizes the encoder of a massively multilingual MT model. |
| Outcome: | The proposed word aligner outperforms existing WA and state-of-the-art non-WA-based methods in token classification tasks. |