Unsupervised Bitext Mining and Translation via Self-Trained Contextual Embeddings (2020.tacl-1)
Copied to clipboard
| Challenge: | Existing methods to extract parallel sentences from unaligned text yield surprisingly good results. |
| Approach: | They propose an unsupervised method to create pseudo-parallel corpora for machine translation (MT) from unaligned text using multilingual BERT to create source and target sentence embeddings for nearest-neighbor search and adapt the model via self-training. |
| Outcome: | The proposed method outperforms existing methods and outperformed previous state-of-the-art methods by boosting translation performance by up to 3.5 BLEU on the WMT’14 French-English and WMT'16 German-English tasks. |
Similar Papers
Unsupervised Multilingual Sentence Embeddings for Parallel Corpus Mining (2020.acl-srw)
Copied to clipboard
| Challenge: | Existing models of multilingual sentence embeddings require large parallel data resources which are not available for low-resource languages. |
| Approach: | They propose an unsupervised method to derive multilingual sentence embeddings using monolingual data. |
| Outcome: | The proposed method improves on two parallel corpus mining tasks and for other languages. |
Bilingual Lexicon Induction via Unsupervised Bitext Construction and Word Alignment (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for bilingual lexicon induction are linear and require simplifying assumptions. |
| Approach: | They propose methods that combine unsupervised bitext mining and unsupervised word alignment to produce higher quality lexicons. |
| Outcome: | The proposed method outperforms the state-of-the-art on the BUCC 2020 task by 14 F1 points . further analysis suggests they are comparable quality . |
Unsupervised Parallel Sentence Extraction with Parallel Segment Detection Helps Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Recent advances in unsupervised bilingual word embeddings make it possible to mine parallel sentences from comparable corpora. |
| Approach: | They propose a strong unsupervised system for parallel sentence mining based on cosine similarities of source and target words . they show that parallel sentences mined from real-life sources improve unsupervised MT . |
| Outcome: | The proposed system improves unsupervised MT on three language pairs. |
UScore: An Effective Approach to Fully Unsupervised Evaluation Metrics for Machine Translation (2023.eacl-main)
Copied to clipboard
| Challenge: | supervised evaluation metrics are not available for machine translation, despite their wide dissemination. |
| Approach: | They develop fully unsupervised evaluation metrics that leverage parallel data and evaluation metric induction. |
| Outcome: | The proposed metrics beat supervised competitors on 4 out of 5 evaluation datasets. |
Filtering and Mining Parallel Data in a Joint Multilingual Space (P18-2)
Copied to clipboard
| Challenge: | Using a cosine distance in a joint multilingual sentence embedding, we filter out noisy parallel data and mine for bitexts in large news collections. |
| Approach: | They propose to learn a joint multilingual sentence embedding and use the distance between sentences in different languages to filter noisy parallel data and to mine for parallel data in large monolingual texts. |
| Outcome: | The proposed approach improves a competitive baseline on the WMT'14 task by 0.3 BLEU by filtering out 25% of the training data. |
CCMatrix: Mining Billions of High-Quality Parallel Sentences on the Web (2021.acl-long)
Copied to clipboard
| Challenge: | Using a curated common crawl corpus, we were able to mine 10.8 billion parallel sentences out of which only 2.9 billions are aligned with English. |
| Approach: | They use 32 snapshots of a curated common crawl corpus totaling 71 billion unique sentences to mine 10.8 billion parallel sentences out of which only 2.9 billions are aligned with English. |
| Outcome: | The proposed system outperforms the best single systems on the WMT’19 test set for English-German/Russian/Chinese and outperformed the best submission at the 2020 WAT workshop. |
Unsupervised Machine Translation in Real-World Scenarios (2022.lrec-1)
Copied to clipboard
Ona de Gibert Bonet, Iakes Goenaga, Jordi Armengol-Estapé, Olatz Perez-de-Viñaspre, Carla Parra Escartín, Marina Sanchez, Mārcis Pinnis, Gorka Labaka, Maite Melero
| Challenge: | a recent study has shown that unsupervised methods rely on monolingual corpora to build MT systems. |
| Approach: | They present the results of the MT4All CEF project using monolingual corpora . they propose to generate bilingual dictionaries and translation models from monolingual data . |
| Outcome: | The proposed method generates bilingual dictionaries and translation models from monolingual corpora . results show that it is comparable to general domain supervised translation . |
Simple and Effective Paraphrastic Similarity from Parallel Translations (P19-1)
Copied to clipboard
| Challenge: | Existing methods for learning paraphrastic sentence embeddings on bitext are expensive and require manual annotation. |
| Approach: | They propose a method that trains paraphrastic sentence embeddings directly from bitext, eliminating the time-consuming step of creating paraphrase corpora. |
| Outcome: | The proposed model outperforms and is faster than state-of-the-art models on cross-lingual tasks. |
Phrase-Based & Neural Unsupervised Machine Translation (D18-1)
Copied to clipboard
| Challenge: | Recent advances in machine translation have reported near human-level performance on several languages, yet their effectiveness strongly relies on the availability of large amounts of parallel sentences. |
| Approach: | They propose two models that leverage a careful initialization of the parameters and denoising effect of language models. |
| Outcome: | The proposed models outperform the current methods on English-French and German-English benchmarks while being simpler and having fewer hyper-parameters. |
Bilingual alignment transfers to multilingual alignment for unsupervised parallel text mining (2022.acl-long)
Copied to clipboard
| Challenge: | a model trained to align only two languages can encode multilingually more aligned representations . a dual-pivot transfer theory is proposed for bilingual training . |
| Approach: | They propose methods for learning cross-lingual sentence representations using paired or unpaired bilingual texts. |
| Outcome: | The proposed models reach the state of the art in unsupervised bitext mining and perform better than multilingually supervised models. |