| Challenge: | Modern neural machine translation models employ a large number of parameters, which leads to serious over-parameterization. |
| Approach: | They propose to prune parameters to improve the model by +0.8 BLEU points and to reallocate them to enhance the ability of modeling low-level lexical information. |
| Outcome: | The pruned parameters improve the model by +0.8 BLEU points and the rejuvenated parameters enhance the ability to model low-level lexical information. |
Similar Papers
Revisiting Low-Resource Neural Machine Translation: A Case Study (P19-1)
Copied to clipboard
| Challenge: | Recent research has shown that neural machine translation models are highly data-inefficient and underperform phrase-based statistical machine translation (PBSMT) in low-resource settings. |
| Approach: | They propose to use auxiliary data to train low-resource neural machine translation systems without auxiliary monolingual or multilingual data. |
| Outcome: | The proposed methods outperform PBSMT and other statistical machine translation models in Korean–English with minimal data. |
Improving Lexical Choice in Neural Machine Translation (N18-1)
Copied to clipboard
| Challenge: | False positives: the output layer rewards frequent words disproportionately, we argue . Falsibles: a model that learns word representations in continuous space tends to translate rare words . |
| Approach: | They propose to fix the norms of both vectors to a constant value and integrate a lexical module which is jointly trained with the rest of the model. |
| Outcome: | The proposed approach achieves improvements of up to +4.3 BLEU surpassing phrase-based translation in nearly all settings. |
Beyond Noise: Mitigating the Impact of Fine-grained Semantic Divergences on Neural Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Prior work treats all types of mismatches between source and target as noise . Consequently, it remains unclear how noisy parallel training samples impact NMT training. |
| Approach: | They propose a divergent-aware NMT framework that uses factors to help NMT recover from the degradation caused by naturally occurring divergences. |
| Outcome: | The proposed framework improves translation quality and model calibration on EN-FR tasks. |
A Simple and Effective Approach to Coverage-Aware Neural Machine Translation (P18-2)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) models are used to solve translation problems using long-term models. |
| Approach: | They propose a method to seek a better balance between model confidence and length preference for Neural Machine Translation. |
| Outcome: | The proposed model improves on Chinese-English and English-German translation tasks. |
Tagged Back-translation Revisited: Why Does It Really Work? (2020.acl-main)
Copied to clipboard
| Challenge: | In this paper, we show that neural machine translation systems trained on large back-translated data overfit some of the characteristics of machine-transcribed texts. |
| Approach: | They propose to add a tag to back-translations to help distinguish back-translated data from original parallel training data. |
| Outcome: | The proposed tag helps the system distinguish back-translated data from original parallel training data and is as effective as a tag in high-resource training. |
Data Rejuvenation: Exploiting Inactive Training Examples for Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Large-scale training datasets make training neural machine translation models difficult. |
| Approach: | They propose to identify inactive training examples which contribute less to the model performance and introduce data rejuvenation to improve NMT models' training. |
| Outcome: | The proposed framework stabilizes and accelerates the training process of NMT models, resulting in models with better generalization capability. |
Addressing Troublesome Words in Neural Machine Translation (D18-1)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) has weaknesses in handling lowfrequency and ambiguous words, which we refer to as troublesome words. |
| Approach: | They propose to use contextual memory to memorize which target words should be produced in which situations to translate troublesome words. |
| Outcome: | The proposed method outperforms baseline models on Chinese-to-English and English-to German translation tasks. |
Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translation (2020.acl-main)
Copied to clipboard
| Challenge: | Existing approaches to improve multilingual neural machine translation (NMT) are weak, and lack robustness to support language pairs with varying typological characteristics. |
| Approach: | They propose to deepen NMT models to support language pairs with varying typological characteristics by random online backtranslation. |
| Outcome: | The proposed approach narrows the performance gap with bilingual models and improves zero-shot performance by 10 BLEU, approaching conventional pivot-based methods. |
Multilingual Neural Machine Translation (2020.coling-tutorials)
Copied to clipboard
| Challenge: | In this tutorial, we will cover the latest advances in NMT to enhance low-resource translation. |
| Approach: | They will cover the latest advances in NMT approaches that leverage multilingualism . they will focus on topics such as language divergence, transfer learning and pivoting . |
| Outcome: | This tutorial will cover the latest advances in NMT to enhance low-resource translation models. |
Salute the Classic: Revisiting Challenges of Machine Translation in the Age of Large Language Models (2025.tacl-1)
Copied to clipboard
| Challenge: | a recent study revisits six core challenges that have influenced the evolution of Neural Machine Translation (NMT) domain mismatch, amount of parallel data, rare word prediction, translation of long sentences and sub-optimal beam search remain challenges in LLMs. |
| Approach: | They revisit core challenges that have acted as benchmarks for progress in NMT . they propose to revisit these challenges and offer insights into their relevance . |
| Outcome: | The proposed models significantly improve translation of sentences containing approximately 80 words, even translating documents up to 512 words. |