Language-aware Interlingua for Multilingual Neural Machine Translation (2020.acl-main)
Copied to clipboard
| Challenge: | Existing multilingual neural machine translation models fail to capture diversity and specificity of different languages, resulting in inferior performance against individual models that are sufficiently trained. |
| Approach: | They propose to integrate a language-aware interlingua into an Encoder-Decoder architecture to learn a semantic representation from the semantic spaces of different languages while allowing for language-specific specialization of a particular language pair. |
| Outcome: | The proposed model achieves remarkable improvements over state-of-the-art multilingual NMT models and produces comparable performance with strong individual models. |
Similar Papers
Multilingual Unsupervised NMT using Shared Encoder and Language-Specific Decoders (P19-1)
Copied to clipboard
| Challenge: | Existing approaches to train multiple languages with a shared encoder and multiple decoders are based on denoising autoencoding of each language and back-translating between English and multiple non-English languages. |
| Approach: | They propose a multilingual unsupervised NMT scheme which trains multiple languages with a shared encoder and multiple decoders. |
| Outcome: | The proposed model performs better than the separately trained bilingual models on monolingual corpora and improves by 1.48 BLEU points on WMT test sets. |
Multilingual Neural Machine Translation (2020.coling-tutorials)
Copied to clipboard
| Challenge: | In this tutorial, we will cover the latest advances in NMT to enhance low-resource translation. |
| Approach: | They will cover the latest advances in NMT approaches that leverage multilingualism . they will focus on topics such as language divergence, transfer learning and pivoting . |
| Outcome: | This tutorial will cover the latest advances in NMT to enhance low-resource translation models. |
Multilingual Neural Machine Translation with Language Clustering (D19-1)
Copied to clipboard
| Challenge: | Existing work on multilingual neural machine translation has been neglected due to its burdensome training process. |
| Approach: | They develop a framework that clusters languages into different groups and trains one multilingual model for each cluster. |
| Outcome: | The proposed model reduces the cost of training and improves translation accuracy. |
A Compact and Language-Sensitive Multilingual Translation Method (P19-1)
Copied to clipboard
| Challenge: | Existing paradigms for multilingual neural machine translation do not make full use of language commonality and parameter sharing. |
| Approach: | They propose a multilingual neural machine translation paradigm with one encoder-decoder model that makes full use of language commonality and parameter sharing. |
| Outcome: | The proposed method outperforms strong standard multilingual translation systems on WMT and IWSLT datasets. |
Multilingual Machine Translation: Closing the Gap between Shared and Language-specific Encoder-Decoders (2021.eacl-main)
Copied to clipboard
| Challenge: | State-of-the-art multilingual machine translation relies on a universal encoder-decoder, which requires retraining the entire system to add new languages. |
| Approach: | They propose an encoder-decoder approach that can be extended to new languages by learning their corresponding modules. |
| Outcome: | The proposed approach outperforms the universal encoder-decoder by 3.28 BLEU points on average while allowing to add new languages without retraining the rest of the modules. |
Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translation (2020.acl-main)
Copied to clipboard
| Challenge: | Existing approaches to improve multilingual neural machine translation (NMT) are weak, and lack robustness to support language pairs with varying typological characteristics. |
| Approach: | They propose to deepen NMT models to support language pairs with varying typological characteristics by random online backtranslation. |
| Outcome: | The proposed approach narrows the performance gap with bilingual models and improves zero-shot performance by 10 BLEU, approaching conventional pivot-based methods. |
Beyond Decoder-only: Large Language Models Can be Good Encoders for Machine Translation (2025.findings-acl)
Copied to clipboard
Yingfeng Luo, Tong Zheng, Yongyu Mu, Bei Li, Qinghong Zhang, Yongqi Gao, Ziqiang Xu, Peinan Feng, Xiaoqian Liu, Tong Xiao, JingBo Zhu
| Challenge: | Recent advances in machine translation have focused on a single pre-trained decoder . encoder-decoder architectures have received relatively little attention in NMT . |
| Approach: | They propose a method that leverages LLMs as MT encoders and pairs them with lightweight decoders to develop universal translation models. |
| Outcome: | The proposed method matches or surpasses baselines in terms of translation quality but achieves 75% reduction in memory footprint of the KV cache. |
Exploiting Deep Representations for Neural Machine Translation (D18-1)
Copied to clipboard
| Challenge: | Neural machine translation models typically implement encoder and decoder as multiple layers, but only the top layers are leveraged in the subsequent process, which misses the opportunity to exploit useful information embedded in other layers. |
| Approach: | They propose to expose all of these signals with layer aggregation and multi-layer attention mechanisms and introduce an auxiliary regularization term to encourage different layers to capture diverse information. |
| Outcome: | The proposed approach exposes all of these signals with layer aggregation and multi-layer attention mechanisms on widely-used translation datasets. |
Revisiting Modularized Multilingual NMT to Meet Industrial Demands (2020.emnlp-main)
Copied to clipboard
| Challenge: | Currently, the complete sharing of parameters for multilingual translation (1-1) is the most popular approach because of its compactness. |
| Approach: | They propose to use a multilingual neural machine translation model that only shares modules among the same languages as 1-1 to satisfy industrial requirements. |
| Outcome: | The proposed model can enjoy the benefits of multi-way training without the capacity bottleneck and low maintainability. |
Do Multilingual Neural Machine Translation Models Contain Language Pair Specific Attention Heads? (2021.findings-acl)
Copied to clipboard
| Challenge: | Recent studies on multilingual representations focus on whether there is an emergence of language-independent representations or whether multilingual models partition their weights among different languages. |
| Approach: | They analyze encoder self-attention and encoder-decoder attention heads in a multilingual neural translation model. |
| Outcome: | The proposed model is based on a multilingual neural translation model with a language-independent representation. |