From Bilingual to Multilingual Neural Machine Translation by Incremental Training (P19-2)
Copied to clipboard
| Challenge: | Existing approaches to multilingual neural machine translation are based on task specific models and the addition of one more language is only possible by retraining the whole system. |
| Approach: | They propose a training schedule that scales to more languages without modification of previous components. |
| Outcome: | The proposed training schedule shows close results to state-of-the-art in the WMT task. |
Similar Papers
Multilingual Machine Translation: Closing the Gap between Shared and Language-specific Encoder-Decoders (2021.eacl-main)
Copied to clipboard
| Challenge: | State-of-the-art multilingual machine translation relies on a universal encoder-decoder, which requires retraining the entire system to add new languages. |
| Approach: | They propose an encoder-decoder approach that can be extended to new languages by learning their corresponding modules. |
| Outcome: | The proposed approach outperforms the universal encoder-decoder by 3.28 BLEU points on average while allowing to add new languages without retraining the rest of the modules. |
Improving Multilingual Neural Machine Translation by Utilizing Semantic and Linguistic Features (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing models do not differentiate between semantic and linguistic features, resulting in the entanglement of knowledge and linguistics within the model. |
| Approach: | They propose to exploit both semantic and linguistic features to enhance multilingual translation by disentangling encoder representations and integrating low-level linguistic encoders. |
| Outcome: | The proposed model improves zero-shot translation while maintaining performance in supervised translation on multilingual datasets. |
Building Multilingual Machine Translation Systems That Serve Arbitrary XY Translations (2022.naacl-main)
Copied to clipboard
| Challenge: | Multilingual Neural Machine Translation (MNMT) systems are often limited to many-to-one directions and suffer from poor performance in one-to one directions. |
| Approach: | They propose to build multilingual machine translation systems that serve arbitrary X-Y directions while leveraging multilinguality with a two-stage training strategy of pretraining and finetuning. |
| Outcome: | The proposed system outperforms baseline bilingual models and pivot translation models in most directions without the need for architecture change or extra data collection. |
Multilingual Neural Machine Translation (2020.coling-tutorials)
Copied to clipboard
| Challenge: | In this tutorial, we will cover the latest advances in NMT to enhance low-resource translation. |
| Approach: | They will cover the latest advances in NMT approaches that leverage multilingualism . they will focus on topics such as language divergence, transfer learning and pivoting . |
| Outcome: | This tutorial will cover the latest advances in NMT to enhance low-resource translation models. |
Learn and Consolidate: Continual Adaptation for Zero-Shot and Multilingual Neural Machine Translation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing multilingual neural machine translation models perform poorly on language pairs with no parallel corpus. |
| Approach: | They propose a two-stage approach that encourages original models to acquire language-agnostic multilingual representations from new data and preserves the model architecture without introducing parameters. |
| Outcome: | The proposed approach improves performance in translation directions where existing models are weak and mitigates degeneration in the well-performing translation directions, offering flexibility in the real-world scenario. |
Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translation (2020.acl-main)
Copied to clipboard
| Challenge: | Existing approaches to improve multilingual neural machine translation (NMT) are weak, and lack robustness to support language pairs with varying typological characteristics. |
| Approach: | They propose to deepen NMT models to support language pairs with varying typological characteristics by random online backtranslation. |
| Outcome: | The proposed approach narrows the performance gap with bilingual models and improves zero-shot performance by 10 BLEU, approaching conventional pivot-based methods. |
Neural Machine Translation for Bilingually Scarce Scenarios: a Deep Multi-Task Learning Approach (N18-1)
Copied to clipboard
| Challenge: | Neural machine translation requires large amount of parallel training text to learn a reasonable quality translation model. |
| Approach: | They propose a multi-task learning approach that leverages monolingual linguistic resources in the source side of a machine translation task. |
| Outcome: | The proposed approach is effective on three translation tasks: English-to-French, English- to-Farsi, and English-à-Vietnamese. |
Massively Multilingual Neural Machine Translation (N19-1)
Copied to clipboard
| Challenge: | Multilingual Neural Machine Translation models support translation from multiple source languages into multiple target languages. |
| Approach: | They perform extensive experiments in training massively multilingual NMT models involving up to 103 distinct languages and 204 translation directions simultaneously. |
| Outcome: | The proposed model outperforms the state-of-the-art in low resource settings while supporting up to 59 languages in 116 translation directions. |
Multi-task Learning for Multilingual Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing multilingual neural machine translation systems rely on bitext training data, which is limited and costly to collect. |
| Approach: | They propose a multi-task learning framework that trains the model with the translation task on bitext data and two denoising tasks on monolingual data. |
| Outcome: | The proposed framework outperforms pre-training models for both NMT and cross-lingual transfer learning NLU tasks. |
Knowledge Transfer in Incremental Learning for Multilingual Neural Machine Translation (2023.acl-long)
Copied to clipboard
| Challenge: | Existing studies focus on overcoming catastrophic forgetting on original language pairs while lacking encouragement to learn new knowledge from incremental learning. |
| Approach: | They propose a knowledge transfer method that can adapt original MNMT models to diverse incremental language pairs by flexibly introducing knowledge from external models into original models, which encourages the models to learn new language pairs. |
| Outcome: | The proposed method outperforms baselines on multiple languages while maintaining performance on original language pairs. |