Improving Zero-Shot Translation by Disentangling Positional Information (2021.acl-long)
Copied to clipboard
| Challenge: | Multilingual neural machine translation has shown the capability of directly translating between language pairs unseen in training, i.e. zero-shot translation. |
| Approach: | They propose to remove residual connections in an encoder layer to reduce the difficulty of generalizing to new translation directions. |
| Outcome: | The proposed model outperforms pivot-based translation in terms of quality and ease of integration of new languages. |
Similar Papers
Rethinking Zero-shot Neural Machine Translation: From a Perspective of Latent Variables (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to achieve zero-shot translation suffer from spurious correlations between output language and language invariant semantics. |
| Approach: | They propose a method that denoizes the autoencoder objective based on pivot language into traditional training objective to improve translation accuracy on zero-shot directions. |
| Outcome: | The proposed method eliminates spurious correlations and outperforms state-of-the-art methods on two benchmark machine translation datasets. |
Improving Zero-Shot Multilingual Translation with Universal Representations and Cross-Mapping (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing model structure and training scheme cannot ensure universal representations and cross-mappings because of lacking explicit constraints. |
| Approach: | They propose a state mover’s distance model to model the difference of the representations output by the encoder and a agreement-based training scheme to minimize the proposed distance to learn universal representations. |
| Outcome: | The proposed model can translate between languages unseen during training, i.e., zero-shot translation. |
Improved Zero-shot Neural Machine Translation via Ignoring Spurious Correlations (P19-1)
Copied to clipboard
| Challenge: | Existing approaches to train a multilingual NMT model for low-resource languages are lacking in terms of number of supervised examples. |
| Approach: | They propose to use decoder pre-training and back-translation to solve the degeneracy problem by analyzing spurious correlations between source and decoded sentences. |
| Outcome: | The proposed methods show significant improvement over the pivot-based approach on three challenging multilingual datasets. |
Adapting to Non-Centered Languages for Zero-shot Multilingual Translation (2022.coling-1)
Copied to clipboard
| Challenge: | Existing studies attributed zero-shot translation to domination of central language, e.g. English, but we supplement this viewpoint with the strict dependence of non-centered languages. |
| Approach: | They propose a language-specific modeling method that adapts to non-centered languages to counteract the instability of zero-shot translation. |
| Outcome: | The proposed method performs better than baselines in centered data conditions and can easily fit non-centered data. |
Decoupled Vocabulary Learning Enables Zero-Shot Translation from Unseen Languages (2024.acl-long)
Copied to clipboard
| Challenge: | Multilingual neural machine translation systems learn to map sentences of different languages into a common representation space. |
| Approach: | They propose a setup where we decouple learning of vocabulary and syntax and train to translate while keeping those word representations frozen. |
| Outcome: | The proposed setup achieves near parity with a supervised setting on the TED domain with varying number of languages seen by the encoder. |
Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translation (2020.acl-main)
Copied to clipboard
| Challenge: | Existing approaches to improve multilingual neural machine translation (NMT) are weak, and lack robustness to support language pairs with varying typological characteristics. |
| Approach: | They propose to deepen NMT models to support language pairs with varying typological characteristics by random online backtranslation. |
| Outcome: | The proposed approach narrows the performance gap with bilingual models and improves zero-shot performance by 10 BLEU, approaching conventional pivot-based methods. |
Monolingual Adapters for Zero-Shot Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing adapter layers are more parameter-efficient and provide better performance than bilingual ones. |
| Approach: | They propose to use monolingual adapter layers instead of bilingual ones to compose them and generalize to unseen language pairs. |
| Outcome: | The proposed adapter layer formalism achieves a median improvement of +2.77 BLEU points over a 20-language multilingual Transformer baseline trained on TED talks. |
Narrowing the Gap between Zero- and Few-shot Machine Translation by Matching Styles (2024.findings-naacl)
Copied to clipboard
Weiting Tan, Haoran Xu, Lingfeng Shen, Shuyue Stella Li, Kenton Murray, Philipp Koehn, Benjamin Van Durme, Yunmo Chen
| Challenge: | Recent work shows that large language models can generalize to machine translation using zero-shot examples with in-context learning. |
| Approach: | They investigate the factors contributing to this gap by matching the writing styles of the target corpus. |
| Outcome: | The proposed methods can be enhanced without the need for parallel demonstration examples. |
Improving Multilingual Neural Machine Translation by Utilizing Semantic and Linguistic Features (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing models do not differentiate between semantic and linguistic features, resulting in the entanglement of knowledge and linguistics within the model. |
| Approach: | They propose to exploit both semantic and linguistic features to enhance multilingual translation by disentangling encoder representations and integrating low-level linguistic encoders. |
| Outcome: | The proposed model improves zero-shot translation while maintaining performance in supervised translation on multilingual datasets. |
Consistency by Agreement in Zero-Shot Neural Machine Translation (N19-1)
Copied to clipboard
| Challenge: | In this paper, we focus on zero-shot generalization—a challenging setup that tests models on translation directions they have not been optimized for at training time. |
| Approach: | They propose a method that allows for a consistent agreement-based training method that encourages the model to produce equivalent translations of parallel sentences in auxiliary languages. |
| Outcome: | The proposed model improves on public zero-shot translation benchmarks without loss of performance on supervised translation directions. |