Sparse and Decorrelated Representations for Stable Zero-shot NMT (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Using a single encoder and decoder for all directions is a popular scheme for multilingual NMT. |
| Approach: | They propose a scheme that uses a single encoder and decoder for all directions . they show that enforcing sparsity and decorrelation on encoder intermediate representations mitigates this problem . |
| Outcome: | The proposed model degenerates when decoding non-English texts into English regardless of the target specifier token. |
Similar Papers
Improved Zero-shot Neural Machine Translation via Ignoring Spurious Correlations (P19-1)
Copied to clipboard
| Challenge: | Existing approaches to train a multilingual NMT model for low-resource languages are lacking in terms of number of supervised examples. |
| Approach: | They propose to use decoder pre-training and back-translation to solve the degeneracy problem by analyzing spurious correlations between source and decoded sentences. |
| Outcome: | The proposed methods show significant improvement over the pivot-based approach on three challenging multilingual datasets. |
Improving Multilingual Translation by Representation and Gradient Regularization (2021.emnlp-main)
Copied to clipboard
| Challenge: | Multilingual Neural Machine Translation models often produce low quality translations, often failing to produce outputs in the right target language. |
| Approach: | They propose a joint approach to regularize NMT models at both representation-level and gradient-level to reduce off-target translation occurrences and improve zero-shot translation performance. |
| Outcome: | The proposed approach reduces off-target translation occurrences and improves zero-shot translation performance by +5.59 and +10.38 BLEU on WMT and OPUS datasets. |
Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translation (2020.acl-main)
Copied to clipboard
| Challenge: | Existing approaches to improve multilingual neural machine translation (NMT) are weak, and lack robustness to support language pairs with varying typological characteristics. |
| Approach: | They propose to deepen NMT models to support language pairs with varying typological characteristics by random online backtranslation. |
| Outcome: | The proposed approach narrows the performance gap with bilingual models and improves zero-shot performance by 10 BLEU, approaching conventional pivot-based methods. |
Rethinking Zero-shot Neural Machine Translation: From a Perspective of Latent Variables (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to achieve zero-shot translation suffer from spurious correlations between output language and language invariant semantics. |
| Approach: | They propose a method that denoizes the autoencoder objective based on pivot language into traditional training objective to improve translation accuracy on zero-shot directions. |
| Outcome: | The proposed method eliminates spurious correlations and outperforms state-of-the-art methods on two benchmark machine translation datasets. |
Adapting to Non-Centered Languages for Zero-shot Multilingual Translation (2022.coling-1)
Copied to clipboard
| Challenge: | Existing studies attributed zero-shot translation to domination of central language, e.g. English, but we supplement this viewpoint with the strict dependence of non-centered languages. |
| Approach: | They propose a language-specific modeling method that adapts to non-centered languages to counteract the instability of zero-shot translation. |
| Outcome: | The proposed method performs better than baselines in centered data conditions and can easily fit non-centered data. |
Improving Zero-shot Multilingual Neural Machine Translation by Leveraging Cross-lingual Consistency Regularization (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to improve zero-shot translation performance by learning language-agnostic representations and maximizing cross-lingual transfer have been proposed. |
| Approach: | They propose a cross-lingual consistency regularization to bridge the representation gap between different languages and boost zero-shot translation performance. |
| Outcome: | The proposed model improves translation performance on low-resource and high-res benchmarks and closes the sentence representation gap and aligns the representation space. |
Modularized Multilingual NMT with Fine-grained Interlingua (2024.naacl-long)
Copied to clipboard
| Challenge: | Neural Machine Translation (MNMT) systems lack layer-sharing to generate interlingua features . however, layer-share structure does not guarantee explicit propagation of language-specific features to respective decoders. |
| Approach: | They propose to share top of language-specific encoder layers to enable interlingua features . their method demonstrates an improved average BLEU score by "+2.90" in En-to-Any directions . |
| Outcome: | The proposed approach improves the BLEU score by "+2.90" in En-to-Any directions and by "+1.06" in zero-shot translation. |
Improving Zero-Shot Cross-Lingual Transfer Learning via Robust Training (2021.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained multilingual language encoders do not precisely align words and phrases across languages. |
| Approach: | They propose a learning strategy for training robust models by drawing connections between adversarial examples and failure cases of zero-shot cross-lingual transfer. |
| Outcome: | The proposed model can achieve good performance even if representations of different languages are not aligned well. |
Consistency by Agreement in Zero-Shot Neural Machine Translation (N19-1)
Copied to clipboard
| Challenge: | In this paper, we focus on zero-shot generalization—a challenging setup that tests models on translation directions they have not been optimized for at training time. |
| Approach: | They propose a method that allows for a consistent agreement-based training method that encourages the model to produce equivalent translations of parallel sentences in auxiliary languages. |
| Outcome: | The proposed model improves on public zero-shot translation benchmarks without loss of performance on supervised translation directions. |
Zero-Resource Neural Machine Translation with Monolingual Pivot Data (D19-56)
Copied to clipboard
| Challenge: | Neural machine translation systems have relied on large amounts of parallel training data between the source and target language. |
| Approach: | They propose methods for generating pseudo-parallel corpora using pivot-language data . they use English as the pivot language to train the zero-shot system . |
| Outcome: | The proposed methods improve the zero-shot neural machine translation system for a high-resource language pair using English as the pivot language. |