Multilingual Neural Machine Translation: Can Linguistic Hierarchies Help? (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Multilingual Neural Machine Translation (MNMT) trains a single model that supports translation between multiple languages . transferring knowledge from a diverse set of languages degrades the translation performance due to negative transfer. |
| Approach: | They propose a hierarchical knowledge distillation approach to train multilingual models . they use typological features and phylogeny to overcome negative transfer issue . |
| Outcome: | The proposed approach avoids negative transfer effect by capitalising on language groups generated according to typological features and phylogeny of languages. |
Similar Papers
Unifying the Convergences in Multilingual Neural Machine Translation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to multilingual neural machine translation are overfitting and inconsistency is ignored . |
| Approach: | They propose a training strategy that picks up language-specific best checkpoints for each language pair to teach the current model on the fly. |
| Outcome: | The proposed training strategy alleviates convergence inconsistency and achieves state-of-the-art on language pairs. |
Knowledge Transfer in Incremental Learning for Multilingual Neural Machine Translation (2023.acl-long)
Copied to clipboard
| Challenge: | Existing studies focus on overcoming catastrophic forgetting on original language pairs while lacking encouragement to learn new knowledge from incremental learning. |
| Approach: | They propose a knowledge transfer method that can adapt original MNMT models to diverse incremental language pairs by flexibly introducing knowledge from external models into original models, which encourages the models to learn new language pairs. |
| Outcome: | The proposed method outperforms baselines on multiple languages while maintaining performance on original language pairs. |
Multilingual Agreement for Multilingual Neural Machine Translation (2021.acl-short)
Copied to clipboard
| Challenge: | Existing models that only use auxiliary languages to encourage multilingual agreement ignore the relationships between different language pairs. |
| Approach: | They propose a multilingual agreement-based method which explicitly models the agreement between different translation directions by randomly substituting some fragments of the source language with their counterpart translations of auxiliary languages. |
| Outcome: | The proposed method improves on the multilingual translation task of 10 language pairs. |
MoNMT: Modularly Leveraging Monolingual and Bilingual Knowledge for Neural Machine Translation (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing models for multi-domain translation tasks only use monolingual data, whereas bilingual data is indispensable for improving the models. |
| Approach: | They propose a modular strategy that facilitates the cooperation of monolingual and bilingual knowledge in translation tasks by avoiding catastrophic forgetting. |
| Outcome: | The proposed model exhibits superior generalization and robustness over the conventional approach. |
Learn and Consolidate: Continual Adaptation for Zero-Shot and Multilingual Neural Machine Translation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing multilingual neural machine translation models perform poorly on language pairs with no parallel corpus. |
| Approach: | They propose a two-stage approach that encourages original models to acquire language-agnostic multilingual representations from new data and preserves the model architecture without introducing parameters. |
| Outcome: | The proposed approach improves performance in translation directions where existing models are weak and mitigates degeneration in the well-performing translation directions, offering flexibility in the real-world scenario. |
Building Multilingual Machine Translation Systems That Serve Arbitrary XY Translations (2022.naacl-main)
Copied to clipboard
| Challenge: | Multilingual Neural Machine Translation (MNMT) systems are often limited to many-to-one directions and suffer from poor performance in one-to one directions. |
| Approach: | They propose to build multilingual machine translation systems that serve arbitrary X-Y directions while leveraging multilinguality with a two-stage training strategy of pretraining and finetuning. |
| Outcome: | The proposed system outperforms baseline bilingual models and pivot translation models in most directions without the need for architecture change or extra data collection. |
Multilingual Neural Machine Translation (2020.coling-tutorials)
Copied to clipboard
| Challenge: | In this tutorial, we will cover the latest advances in NMT to enhance low-resource translation. |
| Approach: | They will cover the latest advances in NMT approaches that leverage multilingualism . they will focus on topics such as language divergence, transfer learning and pivoting . |
| Outcome: | This tutorial will cover the latest advances in NMT to enhance low-resource translation models. |
Multi-task Learning for Multilingual Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing multilingual neural machine translation systems rely on bitext training data, which is limited and costly to collect. |
| Approach: | They propose a multi-task learning framework that trains the model with the translation task on bitext data and two denoising tasks on monolingual data. |
| Outcome: | The proposed framework outperforms pre-training models for both NMT and cross-lingual transfer learning NLU tasks. |
Collective Wisdom: Improving Low-resource Neural Machine Translation using Adaptive Knowledge Distillation (2020.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to train high-quality NMT models in bilingually low-resource scenarios are limited by the scarcity of parallel sentence-pairs. |
| Approach: | They propose to distill the knowledge of teacher models to a single student model by using knowledge distillation. |
| Outcome: | The proposed approach achieves up to +0.9 BLEU score improvements compared to strong baselines. |
Self-Distillation for Model Stacking Unlocks Cross-Lingual NLU in 200+ Languages (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) excel on English NLU tasks, yet struggle to extend their NLU capabilities to underrepresented languages. |
| Approach: | They integrate machine translation models (MT) directly into LLM backbones via sample-efficient self-distillation. |
| Outcome: | The proposed model outperforms translation-test models on 127 low-resource languages. |