Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translation (2020.acl-main)
Copied to clipboard
| Challenge: | Existing approaches to improve multilingual neural machine translation (NMT) are weak, and lack robustness to support language pairs with varying typological characteristics. |
| Approach: | They propose to deepen NMT models to support language pairs with varying typological characteristics by random online backtranslation. |
| Outcome: | The proposed approach narrows the performance gap with bilingual models and improves zero-shot performance by 10 BLEU, approaching conventional pivot-based methods. |
Similar Papers
Multilingual Neural Machine Translation (2020.coling-tutorials)
Copied to clipboard
| Challenge: | In this tutorial, we will cover the latest advances in NMT to enhance low-resource translation. |
| Approach: | They will cover the latest advances in NMT approaches that leverage multilingualism . they will focus on topics such as language divergence, transfer learning and pivoting . |
| Outcome: | This tutorial will cover the latest advances in NMT to enhance low-resource translation models. |
Towards Making the Most of Cross-Lingual Transfer for Zero-Shot Neural Machine Translation (2022.acl-long)
Copied to clipboard
| Challenge: | Existing unsupervised neural machine translation systems can degrade when labeled data is limited. |
| Approach: | They propose a multilingual pretraining and multilingual fine-tuning for facilitating cross-lingual transfer in zero-shot translation using a parallel dataset. |
| Outcome: | The proposed model outperforms state-of-the-art models on many-to-English translation by over 7.2 and 5.0 BLEU. |
A Comparison of Transformer and Recurrent Neural Networks on Multilingual Neural Machine Translation (C18-1)
Copied to clipboard
| Challenge: | Recent studies have shown that multilingual NMT models can handle more than one translation direction with a single system. |
| Approach: | They propose a multilingual neural machine translation model that can handle more than one translation direction with a single system. |
| Outcome: | The proposed model performs well in low-resource settings against bilingual systems. |
Massively Multilingual Neural Machine Translation (N19-1)
Copied to clipboard
| Challenge: | Multilingual Neural Machine Translation models support translation from multiple source languages into multiple target languages. |
| Approach: | They perform extensive experiments in training massively multilingual NMT models involving up to 103 distinct languages and 204 translation directions simultaneously. |
| Outcome: | The proposed model outperforms the state-of-the-art in low resource settings while supporting up to 59 languages in 116 translation directions. |
Improving Multilingual Translation by Representation and Gradient Regularization (2021.emnlp-main)
Copied to clipboard
| Challenge: | Multilingual Neural Machine Translation models often produce low quality translations, often failing to produce outputs in the right target language. |
| Approach: | They propose a joint approach to regularize NMT models at both representation-level and gradient-level to reduce off-target translation occurrences and improve zero-shot translation performance. |
| Outcome: | The proposed approach reduces off-target translation occurrences and improves zero-shot translation performance by +5.59 and +10.38 BLEU on WMT and OPUS datasets. |
Improving Zero-shot Multilingual Neural Machine Translation by Leveraging Cross-lingual Consistency Regularization (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to improve zero-shot translation performance by learning language-agnostic representations and maximizing cross-lingual transfer have been proposed. |
| Approach: | They propose a cross-lingual consistency regularization to bridge the representation gap between different languages and boost zero-shot translation performance. |
| Outcome: | The proposed model improves translation performance on low-resource and high-res benchmarks and closes the sentence representation gap and aligns the representation space. |
Zero-Shot Cross-Lingual Transfer of Neural Machine Translation with Multilingual Pretrained Encoders (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on improving cross-lingual transferability of NMT model is under-explored. |
| Approach: | They propose a model that leverages a multilingual pretrained encoder to improve cross-lingual transferability. |
| Outcome: | The proposed model outperforms mBART and m2m-100 on a zero-shot cross-lingual transfer task. |
Improved Zero-shot Neural Machine Translation via Ignoring Spurious Correlations (P19-1)
Copied to clipboard
| Challenge: | Existing approaches to train a multilingual NMT model for low-resource languages are lacking in terms of number of supervised examples. |
| Approach: | They propose to use decoder pre-training and back-translation to solve the degeneracy problem by analyzing spurious correlations between source and decoded sentences. |
| Outcome: | The proposed methods show significant improvement over the pivot-based approach on three challenging multilingual datasets. |
Rethinking Zero-shot Neural Machine Translation: From a Perspective of Latent Variables (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to achieve zero-shot translation suffer from spurious correlations between output language and language invariant semantics. |
| Approach: | They propose a method that denoizes the autoencoder objective based on pivot language into traditional training objective to improve translation accuracy on zero-shot directions. |
| Outcome: | The proposed method eliminates spurious correlations and outperforms state-of-the-art methods on two benchmark machine translation datasets. |
Revisiting Modularized Multilingual NMT to Meet Industrial Demands (2020.emnlp-main)
Copied to clipboard
| Challenge: | Currently, the complete sharing of parameters for multilingual translation (1-1) is the most popular approach because of its compactness. |
| Approach: | They propose to use a multilingual neural machine translation model that only shares modules among the same languages as 1-1 to satisfy industrial requirements. |
| Outcome: | The proposed model can enjoy the benefits of multi-way training without the capacity bottleneck and low maintainability. |