Multilingual Neural Machine Translation with Deep Encoder and Multiple Shallow Decoders (2021.eacl-main)
Copied to clipboard
| Challenge: | Recent work in multilingual translation has improved translation quality surpassing bilingual baselines using deep transformer models with increased capacity. |
| Approach: | They propose a deep encoder with multiple shallow decoders to reduce inference latency while maintaining translation quality. |
| Outcome: | The proposed model achieves 1.8x speedup on average compared to a standard transformer model with no drop in translation quality. |
Similar Papers
Efficient Inference for Multilingual Neural Machine Translation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Multilingual NMT is an attractive solution for production, but to match bilingual quality, it comes at the cost of larger and slower models. |
| Approach: | They propose to use a shallow decoder with vocabulary filtering to speed up inference . they validate their findings with BLEU and chrF on 380 language pairs . |
| Outcome: | The proposed approach can be used in two 20-language multi-parallel settings. |
Shallow-to-Deep Training for Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results show that deep training is 1:4 faster than training from scratch. |
| Approach: | They propose a shallow-to-deep training method that learns deep models by stacking shallow models. |
| Outcome: | The proposed method is 1:4 faster than training from scratch and achieves BLEU scores of 30:33 and 43:29 on two translation tasks. |
Beyond Decoder-only: Large Language Models Can be Good Encoders for Machine Translation (2025.findings-acl)
Copied to clipboard
Yingfeng Luo, Tong Zheng, Yongyu Mu, Bei Li, Qinghong Zhang, Yongqi Gao, Ziqiang Xu, Peinan Feng, Xiaoqian Liu, Tong Xiao, JingBo Zhu
| Challenge: | Recent advances in machine translation have focused on a single pre-trained decoder . encoder-decoder architectures have received relatively little attention in NMT . |
| Approach: | They propose a method that leverages LLMs as MT encoders and pairs them with lightweight decoders to develop universal translation models. |
| Outcome: | The proposed method matches or surpasses baselines in terms of translation quality but achieves 75% reduction in memory footprint of the KV cache. |
Is Encoder-Decoder Redundant for Neural Machine Translation? (2022.aacl-main)
Copied to clipboard
| Challenge: | Encoder-decoder architecture is widely adopted for sequence-to-sequence modeling tasks. |
| Approach: | They propose to combine bilingual and multilingual translations to train a language model to do translation. |
| Outcome: | The proposed approach performs on par with the baseline encoder-decoder Transformer . the proposed approach is compared with the translation model in the target language . |
Multilingual Machine Translation: Closing the Gap between Shared and Language-specific Encoder-Decoders (2021.eacl-main)
Copied to clipboard
| Challenge: | State-of-the-art multilingual machine translation relies on a universal encoder-decoder, which requires retraining the entire system to add new languages. |
| Approach: | They propose an encoder-decoder approach that can be extended to new languages by learning their corresponding modules. |
| Outcome: | The proposed approach outperforms the universal encoder-decoder by 3.28 BLEU points on average while allowing to add new languages without retraining the rest of the modules. |
Optimizing Transformer for Low-Resource Neural Machine Translation (2020.coling-main)
Copied to clipboard
| Challenge: | Language pairs with limited amounts of parallel data remain a challenge for neural machine translation. |
| Approach: | They propose to optimize a Transformer model for low-resource conditions to improve translation quality by 7.3 BLEU points compared to the default settings. |
| Outcome: | The proposed model improves translation quality up to 7.3 BLEU points compared to the default settings on the IWSLT14 training data compared with the Transformer model. |
Learning Language-Specific Layers for Multilingual Machine Translation (2023.acl-long)
Copied to clipboard
| Challenge: | Multilingual Machine Translation (MNMT) is a promising new approach to improve translation quality between non-English languages. |
| Approach: | They propose a language-specific transformer layer to increase model capacity while keeping computation and parameters constant. |
| Outcome: | The proposed approach improves translation quality by 1.3 chrF (1.5 spBLEU) over not using LSLs on a separate decoder architecture. |
Learning Deep Transformer Models for Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Neural machine translation models have advanced the previous state-of-the-art by learning mappings between sequences via neural networks and attention mechanisms. |
| Approach: | They propose to use layer normalization to pass the combination of previous layers to the next layer to improve the model. |
| Outcome: | The proposed model outperforms the shallow Transformer-Big/Base baseline model on English-German and Chinese-English tasks by 0.4-2.4 BLEU points. |
Improving Multilingual Neural Machine Translation by Utilizing Semantic and Linguistic Features (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing models do not differentiate between semantic and linguistic features, resulting in the entanglement of knowledge and linguistics within the model. |
| Approach: | They propose to exploit both semantic and linguistic features to enhance multilingual translation by disentangling encoder representations and integrating low-level linguistic encoders. |
| Outcome: | The proposed model improves zero-shot translation while maintaining performance in supervised translation on multilingual datasets. |
Breaking Down Multilingual Machine Translation (2022.findings-acl)
Copied to clipboard
| Challenge: | Multilingual training is an essential ingredient in machine translation systems . but it has different effects in different multilingual settings, such as many-to-one, one-tomany and many- to-many learning . |
| Approach: | They compare multilingual training settings with encoders and decoders initialized by multilingual learning . they find important attention heads for each language pair and compare their correlations during inference . |
| Outcome: | The proposed models outperform the best models for high-resource languages and one-to-many models for low-resourced languages. |