Improved Zero-shot Neural Machine Translation via Ignoring Spurious Correlations (P19-1)
Copied to clipboard
| Challenge: | Existing approaches to train a multilingual NMT model for low-resource languages are lacking in terms of number of supervised examples. |
| Approach: | They propose to use decoder pre-training and back-translation to solve the degeneracy problem by analyzing spurious correlations between source and decoded sentences. |
| Outcome: | The proposed methods show significant improvement over the pivot-based approach on three challenging multilingual datasets. |
Similar Papers
Rethinking Zero-shot Neural Machine Translation: From a Perspective of Latent Variables (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to achieve zero-shot translation suffer from spurious correlations between output language and language invariant semantics. |
| Approach: | They propose a method that denoizes the autoencoder objective based on pivot language into traditional training objective to improve translation accuracy on zero-shot directions. |
| Outcome: | The proposed method eliminates spurious correlations and outperforms state-of-the-art methods on two benchmark machine translation datasets. |
Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translation (2020.acl-main)
Copied to clipboard
| Challenge: | Existing approaches to improve multilingual neural machine translation (NMT) are weak, and lack robustness to support language pairs with varying typological characteristics. |
| Approach: | They propose to deepen NMT models to support language pairs with varying typological characteristics by random online backtranslation. |
| Outcome: | The proposed approach narrows the performance gap with bilingual models and improves zero-shot performance by 10 BLEU, approaching conventional pivot-based methods. |
Improving Zero-Shot Translation by Disentangling Positional Information (2021.acl-long)
Copied to clipboard
| Challenge: | Multilingual neural machine translation has shown the capability of directly translating between language pairs unseen in training, i.e. zero-shot translation. |
| Approach: | They propose to remove residual connections in an encoder layer to reduce the difficulty of generalizing to new translation directions. |
| Outcome: | The proposed model outperforms pivot-based translation in terms of quality and ease of integration of new languages. |
Zero-Resource Neural Machine Translation with Monolingual Pivot Data (D19-56)
Copied to clipboard
| Challenge: | Neural machine translation systems have relied on large amounts of parallel training data between the source and target language. |
| Approach: | They propose methods for generating pseudo-parallel corpora using pivot-language data . they use English as the pivot language to train the zero-shot system . |
| Outcome: | The proposed methods improve the zero-shot neural machine translation system for a high-resource language pair using English as the pivot language. |
Improving Zero-shot Multilingual Neural Machine Translation by Leveraging Cross-lingual Consistency Regularization (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to improve zero-shot translation performance by learning language-agnostic representations and maximizing cross-lingual transfer have been proposed. |
| Approach: | They propose a cross-lingual consistency regularization to bridge the representation gap between different languages and boost zero-shot translation performance. |
| Outcome: | The proposed model improves translation performance on low-resource and high-res benchmarks and closes the sentence representation gap and aligns the representation space. |
Towards Making the Most of Cross-Lingual Transfer for Zero-Shot Neural Machine Translation (2022.acl-long)
Copied to clipboard
| Challenge: | Existing unsupervised neural machine translation systems can degrade when labeled data is limited. |
| Approach: | They propose a multilingual pretraining and multilingual fine-tuning for facilitating cross-lingual transfer in zero-shot translation using a parallel dataset. |
| Outcome: | The proposed model outperforms state-of-the-art models on many-to-English translation by over 7.2 and 5.0 BLEU. |
Pivot-based Transfer Learning for Neural Machine Translation between Non-English Languages (D19-1)
Copied to clipboard
| Challenge: | Using parallel corpora, we train a single, direct NMT model for non-English language pairs. |
| Approach: | They propose three ways to increase the relation among source, pivot, and target languages in pre-training . they use additional adapter component to smoothly connect pre-trained encoder and decoder . |
| Outcome: | The proposed methods outperform multilingual models up to +2.6% BLEU in WMT 2019 French-German and German-Czech tasks. |
Adapting to Non-Centered Languages for Zero-shot Multilingual Translation (2022.coling-1)
Copied to clipboard
| Challenge: | Existing studies attributed zero-shot translation to domination of central language, e.g. English, but we supplement this viewpoint with the strict dependence of non-centered languages. |
| Approach: | They propose a language-specific modeling method that adapts to non-centered languages to counteract the instability of zero-shot translation. |
| Outcome: | The proposed method performs better than baselines in centered data conditions and can easily fit non-centered data. |
Consistency by Agreement in Zero-Shot Neural Machine Translation (N19-1)
Copied to clipboard
| Challenge: | In this paper, we focus on zero-shot generalization—a challenging setup that tests models on translation directions they have not been optimized for at training time. |
| Approach: | They propose a method that allows for a consistent agreement-based training method that encourages the model to produce equivalent translations of parallel sentences in auxiliary languages. |
| Outcome: | The proposed model improves on public zero-shot translation benchmarks without loss of performance on supervised translation directions. |
A Comparison of Transformer and Recurrent Neural Networks on Multilingual Neural Machine Translation (C18-1)
Copied to clipboard
| Challenge: | Recent studies have shown that multilingual NMT models can handle more than one translation direction with a single system. |
| Approach: | They propose a multilingual neural machine translation model that can handle more than one translation direction with a single system. |
| Outcome: | The proposed model performs well in low-resource settings against bilingual systems. |