Data Rejuvenation: Exploiting Inactive Training Examples for Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Large-scale training datasets make training neural machine translation models difficult. |
| Approach: | They propose to identify inactive training examples which contribute less to the model performance and introduce data rejuvenation to improve NMT models' training. |
| Outcome: | The proposed framework stabilizes and accelerates the training process of NMT models, resulting in models with better generalization capability. |
Similar Papers
Neural Machine Translation for Agglutinative Languages via Data Rejuvenation (2025.acl-srw)
Copied to clipboard
| Challenge: | Recent years, advances in Neural Machine Translation (NMT) heavily rely on large-scale parallel corpora. |
| Approach: | They propose to combine fine-grained inactive sample identification with target-side rejuvenation to improve translation quality from agglutinative languages. |
| Outcome: | The proposed framework improves on four low-resource agglutinative language tasks. |
On the Sparsity of Neural Machine Translation Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Modern neural machine translation models employ a large number of parameters, which leads to serious over-parameterization. |
| Approach: | They propose to prune parameters to improve the model by +0.8 BLEU points and to reallocate them to enhance the ability of modeling low-level lexical information. |
| Outcome: | The pruned parameters improve the model by +0.8 BLEU points and the rejuvenated parameters enhance the ability to model low-level lexical information. |
Active Learning Approaches to Enhancing Neural Machine Translation (2020.findings-emnlp)
Copied to clipboard
| Challenge: | a limited human translation budget is required to train neural machine translation models. |
| Approach: | They propose to integrate active learning into neural machine translation techniques . they propose a word frequency based acquisition function and an uncertainty based method . |
| Outcome: | The proposed method outperforms other acquisition functions on a limited human translation budget. |
Data Pruning for Efficient Model Pruning in Neural Machine Translation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large-scale pre-trained language models have demonstrated encouraging performance in various NLP tasks at the cost of over-parametrized networks and high memory requirements. |
| Approach: | They combine data pruning with movement pruning for Neural Machine Translation to enable efficient fine-pruning by leveraging cross-entropy scores of individual training instances. |
| Outcome: | The proposed pruning strategy outperforms other pruning methods on a translation task and shows that training cross-entropy scores can reduce the steps required for convergence and training time. |
Towards Robust Neural Machine Translation with Iterative Scheduled Data-Switch Training (2022.coling-1)
Copied to clipboard
Zhongjian Miao, Xiang Li, Liyan Kang, Wen Zhang, Chulun Zhou, Yidong Chen, Bin Wang, Min Zhang, Jinsong Su
| Challenge: | Existing methods on robust neural machine translation (NMT) construct adversarial examples by injecting noise into authentic examples and indiscriminately exploit two types of examples. |
| Approach: | They propose an iterative scheduled data-switch training framework to mitigate this problem by injecting noise into authentic examples and indiscriminately exploiting two types of examples. |
| Outcome: | The proposed model outperforms several competitive benchmarks on four translation benchmarks. |
Pre-training Methods for Neural Machine Translation (2021.acl-tutorials)
Copied to clipboard
| Challenge: | This tutorial provides a comprehensive guide to make the most of pre-training for neural machine translation. |
| Approach: | This tutorial provides a comprehensive guide to make the most of pre-training for neural machine translation. |
| Outcome: | This tutorial explains how to make the most of pre-training for neural machine translation. |
Effective Adversarial Regularization for Neural Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Existing (small) perturbations that induce a critical prediction error in machine learning models are often referred to as adversarial examples. |
| Approach: | They propose to use adversarial perturbations to regularize text classification tasks by adding adversarials to a typical NMT model structure. |
| Outcome: | The proposed method significantly improves performance of NMT models, such as LSTM-based and Transformer-based models. |
Tagged Back-translation Revisited: Why Does It Really Work? (2020.acl-main)
Copied to clipboard
| Challenge: | In this paper, we show that neural machine translation systems trained on large back-translated data overfit some of the characteristics of machine-transcribed texts. |
| Approach: | They propose to add a tag to back-translations to help distinguish back-translated data from original parallel training data. |
| Outcome: | The proposed tag helps the system distinguish back-translated data from original parallel training data and is as effective as a tag in high-resource training. |
Learning to Generalize to More: Continuous Semantic Augmentation for Neural Machine Translation (2022.acl-long)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) tasks require large amounts of parallel data to augment training. |
| Approach: | They propose a data augmentation paradigm that augments each training instance with an adjacency semantic region that could cover adequate variants of literal expression under the same meaning. |
| Outcome: | The proposed paradigm improves on the state-of-the-art in supervised neural machine translation tasks. |
Learning Kernel-Smoothed Machine Translation with Retrieved Examples (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to update deployed models are prone to overfit . however, non-parametric methods are liable to over-fit the retrieved examples . |
| Approach: | They propose to learn Kernel-Smoothed Translation with Example Retrieval (KSTER) this approach allows users to adapt models to emerging cases without retraining . |
| Outcome: | The proposed approach achieves 1.1 to 1.5 BLEU scores over existing methods without retraining . the proposed model is released on https://github.com/jiangqn/KSTER. |