Multilingual Unsupervised NMT using Shared Encoder and Language-Specific Decoders (P19-1)
Copied to clipboard
| Challenge: | Existing approaches to train multiple languages with a shared encoder and multiple decoders are based on denoising autoencoding of each language and back-translating between English and multiple non-English languages. |
| Approach: | They propose a multilingual unsupervised NMT scheme which trains multiple languages with a shared encoder and multiple decoders. |
| Outcome: | The proposed model performs better than the separately trained bilingual models on monolingual corpora and improves by 1.48 BLEU points on WMT test sets. |
Similar Papers
Language-aware Interlingua for Multilingual Neural Machine Translation (2020.acl-main)
Copied to clipboard
| Challenge: | Existing multilingual neural machine translation models fail to capture diversity and specificity of different languages, resulting in inferior performance against individual models that are sufficiently trained. |
| Approach: | They propose to integrate a language-aware interlingua into an Encoder-Decoder architecture to learn a semantic representation from the semantic spaces of different languages while allowing for language-specific specialization of a particular language pair. |
| Outcome: | The proposed model achieves remarkable improvements over state-of-the-art multilingual NMT models and produces comparable performance with strong individual models. |
Multilingual Machine Translation: Closing the Gap between Shared and Language-specific Encoder-Decoders (2021.eacl-main)
Copied to clipboard
| Challenge: | State-of-the-art multilingual machine translation relies on a universal encoder-decoder, which requires retraining the entire system to add new languages. |
| Approach: | They propose an encoder-decoder approach that can be extended to new languages by learning their corresponding modules. |
| Outcome: | The proposed approach outperforms the universal encoder-decoder by 3.28 BLEU points on average while allowing to add new languages without retraining the rest of the modules. |
Modularized Multilingual NMT with Fine-grained Interlingua (2024.naacl-long)
Copied to clipboard
| Challenge: | Neural Machine Translation (MNMT) systems lack layer-sharing to generate interlingua features . however, layer-share structure does not guarantee explicit propagation of language-specific features to respective decoders. |
| Approach: | They propose to share top of language-specific encoder layers to enable interlingua features . their method demonstrates an improved average BLEU score by "+2.90" in En-to-Any directions . |
| Outcome: | The proposed approach improves the BLEU score by "+2.90" in En-to-Any directions and by "+1.06" in zero-shot translation. |
An Effective Approach to Unsupervised Machine Translation (P19-1)
Copied to clipboard
| Challenge: | a recent research line has managed to train both unsupervised and unsupervised machine translation systems using monolingual corpora only. |
| Approach: | They propose to use monolingual corpora to train both unsupervised and unsupervised machine translation systems. |
| Outcome: | The proposed system achieves 22.5 BLEU points in English-to-German WMT 2014, 5.5 points more than the previous best unsupervised system, and 0.5 points more in the (supervised) shared task winner back in 2014. |
Unsupervised Neural Machine Translation with Weight Sharing (P18-1)
Copied to clipboard
| Challenge: | Unsupervised neural machine translation (NMT) is a new approach for machine translation . the model uses only one shared encoder to map pairs of sentences from different languages to a shared-latent space . |
| Approach: | They propose an unsupervised approach which trains the model without labeling data . they propose two independent encoders but share some partial weights to extract high-level representations of input sentences. |
| Outcome: | The proposed approach achieves significant improvements on English-German, English-French and Chinese-to-English translation tasks. |
Multilingual Unsupervised Neural Machine Translation with Denoising Adapters (2021.emnlp-main)
Copied to clipboard
| Challenge: | Multilingual unsupervised machine translation is a computationally expensive and hard to tune approach . auxiliary parallel data is used to train translation systems from monolingual data . |
| Approach: | They propose to use auxiliary parallel language pairs to train unsupervised machine translations . they propose to add auxiliary languages to pre-trained mBART-50 models with denoising adapters . |
| Outcome: | The proposed approach is on-par with back-translation and allows adding unseen languages incrementally. |
Cross-lingual Supervision Improves Unsupervised Neural Machine Translation (2021.naacl-industry)
Copied to clipboard
| Challenge: | Existing models that use only monolingual data have not been fully duplicated in the vast majority of language pairs, especially for zero-source languages. |
| Approach: | They propose to leverage the corpus from En-Fr and En-De to collectively train the translation from one language into many languages under one model. |
| Outcome: | The proposed model significantly improves translation quality with a big margin in the benchmark unsupervised translation tasks and achieves comparable performance to supervised NMT. |
Multilingual Neural Machine Translation with Language Clustering (D19-1)
Copied to clipboard
| Challenge: | Existing work on multilingual neural machine translation has been neglected due to its burdensome training process. |
| Approach: | They develop a framework that clusters languages into different groups and trains one multilingual model for each cluster. |
| Outcome: | The proposed model reduces the cost of training and improves translation accuracy. |
Exploring Unsupervised Pretraining Objectives for Machine Translation (2021.findings-acl)
Copied to clipboard
| Challenge: | Unsupervised cross-lingual pretraining has significantly reduced the need for large parallel data. |
| Approach: | They compare unsupervised cross-lingual pretraining with masking and reconstructing inputs in the decoder to produce real sentences. |
| Outcome: | The proposed methods produce inputs resembling real (full) sentences, by reordering and replacing words based on their context. |
Pivot-based Transfer Learning for Neural Machine Translation between Non-English Languages (D19-1)
Copied to clipboard
| Challenge: | Using parallel corpora, we train a single, direct NMT model for non-English language pairs. |
| Approach: | They propose three ways to increase the relation among source, pivot, and target languages in pre-training . they use additional adapter component to smoothly connect pre-trained encoder and decoder . |
| Outcome: | The proposed methods outperform multilingual models up to +2.6% BLEU in WMT 2019 French-German and German-Czech tasks. |