Neuron-Level Language Tag Injection Improves Zero-Shot Translation Performance (2025.acl-srw)
Copied to clipboard
| Challenge: | Language tagging is a method that trains models on specific language directions . injection is based on a language token embedded in the input layer . |
| Approach: | They propose a method whereby source and target inputs are prefixed with a unique language token and inject it into the input of every linear layer. |
| Outcome: | The proposed method improves translation performance with up to 2+ BLEU score point gain for certain language directions in a multilingual dataset. |
Similar Papers
Language Tags Matter for Zero-Shot Neural Machine Translation (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on multilingual machine translation have ignored the importance of LTs. |
| Approach: | They propose to use language tag (LT) strategies to indicate translation directions in MNMT to enhance consistency and alleviate off-target issues in zero-shot directions. |
| Outcome: | The proposed model could translate between unsupervised languages and achieve a +8 BLEU score difference over other LT strategies in translation tasks. |
LCS: A Language Converter Strategy for Zero-Shot Neural Machine Translation (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing LT strategies cannot indicate the desired target language on zero-shot translation, i.e., the off-target issue. |
| Approach: | They propose a language converter strategy that embeds the target language into the top encoder layers to mitigate confusion in the encoder and ensures stable language indication for the decoder. |
| Outcome: | The proposed language converter strategy significantly mitigates off-target issue on multiUN, TED, and OPUS-100 datasets. |
An Empirical Investigation of Word Alignment Supervision for Zero-Shot Multilingual Neural Machine Translation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent work has highlighted several flaws of MNMT models in zero-shot scenarios where language labels are ignored and the wrong language is generated. |
| Approach: | They propose to combine explicit alignment to language labels with word alignment supervision to improve zero-shot translations. |
| Outcome: | The proposed model improves on three multilingual MT benchmarks. |
Learn and Consolidate: Continual Adaptation for Zero-Shot and Multilingual Neural Machine Translation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing multilingual neural machine translation models perform poorly on language pairs with no parallel corpus. |
| Approach: | They propose a two-stage approach that encourages original models to acquire language-agnostic multilingual representations from new data and preserves the model architecture without introducing parameters. |
| Outcome: | The proposed approach improves performance in translation directions where existing models are weak and mitigates degeneration in the well-performing translation directions, offering flexibility in the real-world scenario. |
Registering Source Tokens to Target Language Spaces in Multilingual Neural Machine Translation (2025.acl-long)
Copied to clipboard
| Challenge: | Multilingual neural machine translation (MNMT) aims for arbitrary translations across multiple languages. |
| Approach: | They propose a method that inserts a set of tokens specifying the target language into the input sequence between the source and target tokens. |
| Outcome: | The proposed method outperforms existing models on a large-scale benchmark. |
The Effects of Language Token Prefixing for Multilingual Machine Translation (2022.aacl-short)
Copied to clipboard
| Challenge: | In recent years, the field has moved towards large neural models either translating from or into many languages. |
| Approach: | They propose to prefix language tokens onto a source or target sequence to improve translation performance. |
| Outcome: | The proposed methods improve translation performance and source side prefixes improve translation. |
Improving Zero-Shot Translation by Disentangling Positional Information (2021.acl-long)
Copied to clipboard
| Challenge: | Multilingual neural machine translation has shown the capability of directly translating between language pairs unseen in training, i.e. zero-shot translation. |
| Approach: | They propose to remove residual connections in an encoder layer to reduce the difficulty of generalizing to new translation directions. |
| Outcome: | The proposed model outperforms pivot-based translation in terms of quality and ease of integration of new languages. |
Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translation (2020.acl-main)
Copied to clipboard
| Challenge: | Existing approaches to improve multilingual neural machine translation (NMT) are weak, and lack robustness to support language pairs with varying typological characteristics. |
| Approach: | They propose to deepen NMT models to support language pairs with varying typological characteristics by random online backtranslation. |
| Outcome: | The proposed approach narrows the performance gap with bilingual models and improves zero-shot performance by 10 BLEU, approaching conventional pivot-based methods. |
Pre-trained Language Models Can be Fully Zero-Shot Learners (2023.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to pre-trained language models require fine-tuning on labeled datasets or manually constructing proper prompts. |
| Approach: | They propose a nonparametric prompting PLM for fully zero-shot language understanding . they compare it to previous methods for text classification and text entailment . |
| Outcome: | The proposed method outperforms previous methods on diverse tasks. |
Towards a Better Understanding of Variations in Zero-Shot Neural Machine Translation Performance (2023.emnlp-main)
Copied to clipboard
| Challenge: | Prior work has investigated causes of poor zero-shot performance, but new study suggests it does not exhibit poor zero shot capability. |
| Approach: | They propose to investigate the presence of significant variations in zero-shot performance . target-side translation quality is most influential factor, with vocabulary overlap impacting zero- shot capabilities . |
| Outcome: | The results show that the target side translation quality is the most influential factor . linguistic properties, such as language family and writing system, play a role . |