| Challenge: | Recent advances in machine translation have reported near human-level performance on several languages, yet their effectiveness strongly relies on the availability of large amounts of parallel sentences. |
| Approach: | They propose two models that leverage a careful initialization of the parameters and denoising effect of language models. |
| Outcome: | The proposed models outperform the current methods on English-French and German-English benchmarks while being simpler and having fewer hyper-parameters. |
Similar Papers
An Effective Approach to Unsupervised Machine Translation (P19-1)
Copied to clipboard
| Challenge: | a recent research line has managed to train both unsupervised and unsupervised machine translation systems using monolingual corpora only. |
| Approach: | They propose to use monolingual corpora to train both unsupervised and unsupervised machine translation systems. |
| Outcome: | The proposed system achieves 22.5 BLEU points in English-to-German WMT 2014, 5.5 points more than the previous best unsupervised system, and 0.5 points more in the (supervised) shared task winner back in 2014. |
Unsupervised Statistical Machine Translation (D18-1)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) systems can be trained from monolingual corpora without supervision. |
| Approach: | They propose a phrase-based approach that trains from monolingual corpora . their method is based on phrase-driven Statistical Machine Translation (SMT) they propose to train NMT systems without supervision from monolinguistic corpors . |
| Outcome: | The proposed approach improves on the existing supervised systems by combining a phrase table with an n-gram language model and fine-tuning hyperparameters through an unsupervised MERT variant. |
Multilingual Unsupervised Neural Machine Translation with Denoising Adapters (2021.emnlp-main)
Copied to clipboard
| Challenge: | Multilingual unsupervised machine translation is a computationally expensive and hard to tune approach . auxiliary parallel data is used to train translation systems from monolingual data . |
| Approach: | They propose to use auxiliary parallel language pairs to train unsupervised machine translations . they propose to add auxiliary languages to pre-trained mBART-50 models with denoising adapters . |
| Outcome: | The proposed approach is on-par with back-translation and allows adding unseen languages incrementally. |
A Multilingual View of Unsupervised Machine Translation (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Empirically, we show that our approach results in higher BLEU scores over state-of-the-art unsupervised models on the WMT’14 English-French, WMT'16 English-German, and WMT‘16 English–Romanian datasets in most directions. |
| Approach: | They propose a probabilistic framework for multilingual neural machine translation that encompasses supervised and unsupervised setups, focusing on unsupervised translation. |
| Outcome: | The proposed framework achieves higher BLEU scores than state-of-the-art unsupervised models on the WMT’14 English-French, WMT'16 English-German, and WMT‘16 English–Romanian datasets in most directions. |
Improving Machine Translation with Phrase Pair Injection and Corpus Filtering (2022.emnlp-main)
Copied to clipboard
| Challenge: | In this paper, we show that the combination of Phrase Pair Injection and Corpus Filtering boosts the performance of Neural Machine Translation systems. |
| Approach: | They propose to combine Phrase Pair Injection and Corpus Filtering to boost performance of Neural Machine Translation systems. |
| Outcome: | The proposed method improves machine translation models on low-resource language pairs . BLEU score improves over models trained with whole pseudo-parallel corpus augmented with parallel corpus. |
Neural Machine Translation for Bilingually Scarce Scenarios: a Deep Multi-Task Learning Approach (N18-1)
Copied to clipboard
| Challenge: | Neural machine translation requires large amount of parallel training text to learn a reasonable quality translation model. |
| Approach: | They propose a multi-task learning approach that leverages monolingual linguistic resources in the source side of a machine translation task. |
| Outcome: | The proposed approach is effective on three translation tasks: English-to-French, English- to-Farsi, and English-à-Vietnamese. |
Data augmentation using back-translation for context-aware neural machine translation (D19-65)
Copied to clipboard
| Challenge: | A single sentence does not always convey information that is enough to translate it into other languages. |
| Approach: | They obtain large-scale pseudo parallel corpora by back-translating monolingual data and examine their impact on translation accuracy. |
| Outcome: | The large-scale pseudo parallel corpora obtained by back-translating monolingual data showed that the model trained with small parallel corporeals and large-sized pseudo parallels improved translation accuracy. |
Breaking the Corpus Bottleneck for Context-Aware Neural Machine Translation with Cross-Task Pre-training (2021.acl-long)
Copied to clipboard
| Challenge: | Context-aware neural machine translation (NMT) remains challenging due to the lack of large-scale document-level parallel corpora. |
| Approach: | They propose to use large-scale parallel datasets and source-side monolingual documents to improve context-aware neural machine translation. |
| Outcome: | The proposed model can be used to translate both sentences and documents on four translation tasks. |
Learning Source Phrase Representations for Neural Machine Translation (2020.acl-main)
Copied to clipboard
| Challenge: | Existing approaches to machine translation have been shown to be effective for long sentences . however, the attentional network can't capture long-distance dependencies . |
| Approach: | They propose a multi-head attention mechanism which generates phrase representations from token representations and incorporates them into the Transformer translation model to enhance its ability to capture long-distance relationships. |
| Outcome: | The proposed model can be computed in parallel and improves on the WMT 14 tasks. |
Unsupervised Neural Machine Translation with Universal Grammar (2021.emnlp-main)
Copied to clipboard
| Challenge: | Unsupervised machine translation relies on parallel corpora for training, but performance still lags behind traditional supervised machine translators. |
| Approach: | They propose to leverage shared grammar clues to provide more explicit language parallel signals to enhance the training of unsupervised machine translation models. |
| Outcome: | The proposed models improve on a common language pair training task in English and german, and use embedding alignments and pretrained language models to synthesize pseudo parallel corpora. |