| Challenge: | Multilingual training is an essential ingredient in machine translation systems . but it has different effects in different multilingual settings, such as many-to-one, one-tomany and many- to-many learning . |
| Approach: | They compare multilingual training settings with encoders and decoders initialized by multilingual learning . they find important attention heads for each language pair and compare their correlations during inference . |
| Outcome: | The proposed models outperform the best models for high-resource languages and one-to-many models for low-resourced languages. |
Similar Papers
Multilingual Machine Translation: Closing the Gap between Shared and Language-specific Encoder-Decoders (2021.eacl-main)
Copied to clipboard
| Challenge: | State-of-the-art multilingual machine translation relies on a universal encoder-decoder, which requires retraining the entire system to add new languages. |
| Approach: | They propose an encoder-decoder approach that can be extended to new languages by learning their corresponding modules. |
| Outcome: | The proposed approach outperforms the universal encoder-decoder by 3.28 BLEU points on average while allowing to add new languages without retraining the rest of the modules. |
Revisiting Machine Translation for Cross-lingual Classification (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent work in cross-lingual learning has pivoted around multilingual models, which are typically pretrained on unlabeled corpora in multiple languages using some form of language modeling objective. |
| Approach: | They propose to use a stronger machine translation system to mitigat mismatch between training on original text and running inference on machine translated text. |
| Outcome: | The proposed approach is highly task dependent and calls into question the dominance of multilingual models for cross-lingual classification. |
Balancing Training for Multilingual Neural Machine Translation (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods to train multilingual machine translation models are imbalanced and heterogeneous data is wildly varying. |
| Approach: | They propose a method that automatically learns how to weight training data through a data scorer that is optimized to maximize performance on all test languages. |
| Outcome: | The proposed method outperforms baselines on two sets of languages under one-to-many and many-to-1 MT settings and offers flexible control over which languages are optimized. |
Three Strategies to Improve One-to-Many Multilingual Translation (D18-1)
Copied to clipboard
| Challenge: | Existing studies show that one-to-many multilingual translation cannot perform on par with the individually trained models. |
| Approach: | They propose to exploit unique initial states for target languages and language-dependent positional embeddings to create hidden cells of the encoder to achieve comparable or even better performance than individually trained models. |
| Outcome: | The proposed methods achieve comparable or even better performance than the individually trained models. |
Multilingual Translation from Denoising Pre-Training (2021.findings-acl)
Copied to clipboard
Yuqing Tang, Chau Tran, Xian Li, Peng-Jen Chen, Naman Goyal, Vishrav Chaudhary, Jiatao Gu, Angela Fan
| Challenge: | Recent work shows potential of training one model for multilingual machine translation . but little has been explored on the potential to combine denoising pretraining with multilingual translation in a single model. |
| Approach: | They propose to combine denoising pretraining with multilingual machine translation in a single model. |
| Outcome: | The proposed model improves over models trained from scratch and bilingually for translation into English. |
Many-to-English Machine Translation Tools, Data, and Pretrained Models (2021.acl-demo)
Copied to clipboard
| Challenge: | Commercial translation systems support only one hundred languages or fewer . commercial translation systems do not make these models available for transfer to low resource languages . |
| Approach: | They propose a multilingual neural machine translation model that can translate from 500 source languages to English. |
| Outcome: | The proposed model can translate from 500 source languages to English, or be used as a parent model for low-resource languages. |
Can Machine Translation Bridge Multilingual Pretraining and Cross-lingual Transfer Learning? (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing models that pretrain for cross-lingual tasks do not improve cross-linguistic learning. |
| Approach: | They propose to employ machine translation as a continued training objective to enhance language representation learning by bridging multilingual pretraining and cross-lingual applications. |
| Outcome: | The proposed model performance is compared with existing models and their latent representations. |
To Translate or Not to Translate: A Systematic Investigation of Translation-Based Cross-Lingual Transfer to Low-Resource Languages (2024.naacl-long)
Copied to clipboard
| Challenge: | XLT with multilingual language models is superfluous, says a new study . mBERT, XLM-R and mT5 are effective for cross-lingual transfer, authors say . |
| Approach: | They propose to use multilingual language models to improve cross-lingual transfer (XLT) they propose to add reliable translations to training data for XLT even for non-MT languages . |
| Outcome: | The proposed approaches outperform zero-shot XLT with mLMs, the authors show . the authors believe their findings warrant a broader inclusion of more robust translation-based baselines in XL research. |
The Role of Mixed-Language Documents for Multilingual Large Language Model Pretraining (2026.acl-long)
Copied to clipboard
Jiandong Shao, Raphael Tang, Crystina Zhang, Karin Sevegnani, Pontus Stenetorp, Jianfei Yang, Yao Lu
| Challenge: | Existing research suggests that multilingual large language models can achieve impressive cross-lingual understanding despite largely monolingual pretraining. |
| Approach: | They compare a monolingual-only corpus with a standard web corpus that removes all multilingual documents and then retrain the models from scratch under controlled conditions. |
| Outcome: | The results show that removing bilingual data causes translation performance to drop 56% in BLEU, whereas code-switching contributes minimally. |
Recipes for Adapting Pre-trained Monolingual and Multilingual Models to Machine Translation (2021.eacl-main)
Copied to clipboard
| Challenge: | Recent advances in machine translation (MT) have improved performance on low-resource language pairs. |
| Approach: | They propose to freeze most BART parameters and add new ones to fine-tune a model trained on MT. |
| Outcome: | The proposed model outperforms naive fine-tuning on Vietnamese to English on a training set for Vietnamese to Vietnamese . the proposed model is able to fine- tune on smaller datasets while still maintaining the same model performance. |