| Challenge: | Existing methods for learning paraphrastic sentence embeddings on bitext are expensive and require manual annotation. |
| Approach: | They propose a method that trains paraphrastic sentence embeddings directly from bitext, eliminating the time-consuming step of creating paraphrase corpora. |
| Outcome: | The proposed model outperforms and is faster than state-of-the-art models on cross-lingual tasks. |
Similar Papers
Paraphrastic Representations at Scale (2022.emnlp-demos)
Copied to clipboard
| Challenge: | a new system allows users to train their own state-of-the-art paraphrastic sentence representations in a variety of languages. |
| Approach: | They propose a system that allows users to train their own paraphrastic sentence representations in a variety of languages. |
| Outcome: | The proposed models outperform previous models on monolingual and cross-lingual tasks and can be used on CPUs with little difference in inference speed. |
Comparative Study of Sentence Embeddings for Contextual Paraphrasing (2020.lrec-1)
Copied to clipboard
| Challenge: | Paraphrasing is an important aspect of natural-language generation that can produce more variety in the way specific content is presented. |
| Approach: | They propose to use contextual paraphrasing to capture the meaning of a sentence while performing dialogue act clustering. |
| Outcome: | The proposed task combines paraphrases with dialogue act clustering to capture such contextual paraphrasing. |
Exploring Multilingual Syntactic Sentence Representations (D19-55)
Copied to clipboard
| Challenge: | Recent studies on language models that learn syntactic information focus on learning the semantic structures of language. |
| Approach: | They propose to use a multilingual parallel corpus augmented by universal part-of-speech tags to learn syntactic sentence embeddings. |
| Outcome: | The proposed method performs better than state-of-the-art language models in low-resource languages. |
A Multi-task Approach to Learning Multilingual Representations (P18-2)
Copied to clipboard
| Challenge: | Using a multi-task model, we learn word and sentence embeddings in a single task. |
| Approach: | They propose a multi-task modeling approach that trains a skip-gram model and a cross-lingual sentence similarity model to learn word and sentence embeddings together. |
| Outcome: | The proposed model can learn word and sentence embeddings in a multilingual distributed representations of text using a cross-lingual sentence similarity model. |
Improving Simultaneous Translation by Incorporating Pseudo-References with Fewer Reorderings (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing systems for simultaneous translation are still trained on full-sentence bitexts due to the abundance of unnecessary long-distance reorderings. |
| Approach: | They propose to rewrite target side of existing full-sentence corpora into simultaneous-style translation by adding generated pseudo-references to the target side. |
| Outcome: | Experiments on ZhEn and JaEn simultaneous translation show that the proposed method improves on existing full-sentence corpora. |
Improving Multi-lingual Alignment Through Soft Contrastive Learning (2024.naacl-srw)
Copied to clipboard
| Challenge: | Existing methods to train multi-lingual sentence embeddings ruins the mono-lingual space. |
| Approach: | They propose a method to align multi-lingual embeddings based on similarity of sentences measured by a pre-trained mono-lingual teacher model. |
| Outcome: | The proposed method outperforms existing multi-lingual embeddings including LaBSE on five languages and on a translation pair for Tatoeba dataset. |
ParaNMT-50M: Pushing the Limits of Paraphrastic Sentence Embeddings with Millions of Machine Translations (P18-1)
Copied to clipboard
| Challenge: | Using neural machine translation, we generate more than 50 million sentential paraphrase pairs from a large parallel corpus. |
| Approach: | They use a dataset of more than 50 million English-English sentential paraphrase pairs to generate them automatically using neural machine translation. |
| Outcome: | The proposed dataset outperforms all supervised systems on every SemEval semantic textual similarity competition and shows how it can be used for paraphrase generation. |
Cross-lingual Sentence Embedding using Multi-Task Learning (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing multilingual sentence embedding models require large parallel corpora to learn efficiently, limiting their scope. |
| Approach: | They propose a sentence embedding framework based on an unsupervised loss function . they capture semantic similarity and relatedness between sentences using a multi-task loss function. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on STS, BUCC and Tatoeba benchmarks and on a monolingual benchmark. |
Filtering and Mining Parallel Data in a Joint Multilingual Space (P18-2)
Copied to clipboard
| Challenge: | Using a cosine distance in a joint multilingual sentence embedding, we filter out noisy parallel data and mine for bitexts in large news collections. |
| Approach: | They propose to learn a joint multilingual sentence embedding and use the distance between sentences in different languages to filter noisy parallel data and to mine for parallel data in large monolingual texts. |
| Outcome: | The proposed approach improves a competitive baseline on the WMT'14 task by 0.3 BLEU by filtering out 25% of the training data. |
LuxEmbedder: A Cross-Lingual Approach to Enhanced Luxembourgish Sentence Embeddings (2025.coling-main)
Copied to clipboard
| Challenge: | Sentence embedding models are limited for many low-resource languages, including Luxembourgish. |
| Approach: | They propose to use Luxembourgish as an enhanced sentence embedding model with strong cross-lingual capabilities to address this issue. |
| Outcome: | The proposed model can embed Luxembourgish sentences better than high-resource languages. |