Uncertainty-Aware Curriculum Learning for Neural Machine Translation (2020.acl-main)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) has proven to be facilitated by curriculum learning which presents examples in an easy-to-hard order at different training stages. |
| Approach: | They propose to use an uncertainty-aware curriculum learning approach to assess data difficulty and model competence to provide examples in an easy-to-hard order at different training stages. |
| Outcome: | The proposed approach outperforms baseline and related methods on translation quality and convergence speed. |
Similar Papers
Token-wise Curriculum Learning for Neural Machine Translation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing curriculum learning approaches to Neural Machine Translation (NMT) require sampling sufficient amounts of “easy” samples from training data at the early stage of training. |
| Approach: | They propose a token-wise curriculum learning approach that creates sufficient amounts of easy samples from training data. |
| Outcome: | The proposed approach outperforms baselines on five language pairs on low-resource languages. |
Data Selection Curriculum for Neural Machine Translation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Neural Machine Translation models are typically trained on heterogeneous data that are concatenated and randomly shuffled. |
| Approach: | They propose a two-stage curriculum training framework where a NMT model is fine-tuned on subsets of data, selected by deterministic scoring and online scoring. |
| Outcome: | The proposed framework improves on six language pairs comprising low- and high-resource languages and shows up to +2.2 BLEU improvement and faster convergence. |
Uncertainty-Aware Semantic Augmentation for Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for neural machine translation only observe one source sentence at training time . this discrepancy in data distribution leads to a formidable learning challenge . |
| Approach: | They propose an uncertainty-aware semantic augmentation approach to capture universal semantic information among multiple source sentences and enhance hidden representations with this information. |
| Outcome: | The proposed approach outperforms baseline and existing methods on translation tasks. |
Self-Paced Learning for Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have shown that the training of neural machine translation (NMT) rely on the quality of artificial schedule drawn up with the handcrafted features, e.g. sentence length or word rarity. |
| Approach: | They propose to train NMT model using a self-paced learning approach that allows it to quantify the learning confidence over training examples and flexibly govern its learning via regulating the loss in each iteration step. |
| Outcome: | The proposed model outperforms baseline models and those trained with human-designed curricula on translation quality and convergence speed. |
Competence-based Curriculum Learning for Neural Machine Translation (N19-1)
Copied to clipboard
| Challenge: | Existing NMT systems require specialized heuristics and large batch sizes. |
| Approach: | They propose a curriculum learning framework for NMT that reduces training time and costs . framework consists of a principled way of deciding which training samples are shown to the model . |
| Outcome: | The proposed framework can reduce training time and improve performance of recurrent neural network models and Transformers. |
Norm-Based Curriculum Learning for Neural Machine Translation (2020.acl-main)
Copied to clipboard
| Challenge: | Experimental results show that the proposed method outperforms strong baselines in terms of BLEU score (+1.17/+1.56) and training speedup (2.22x/3.33x). |
| Approach: | They propose a norm-based curriculum learning method that measures difficulty, competence and weight of a sentence in a word embedding. |
| Outcome: | The proposed method outperforms baselines in terms of BLEU score (+1.17/+1.56) and training speedup (2.22x/3.33x). |
Reinforcement Learning based Curriculum Optimization for Neural Machine Translation (N19-1)
Copied to clipboard
| Challenge: | a heterogeneous training dataset can vary in characteristics such as domain, translation quality, and degree of difficulty. |
| Approach: | They propose to use reinforcement learning to learn an optimal curriculum for NMT training . they find it can beat uniform baselines and hand-designed, state-of-the-art curricula . |
| Outcome: | The proposed approach beats baselines and hand-designed curricula on English-to-French datasets. |
Measuring Uncertainty in Neural Machine Translation with Similarity-Sensitive Entropy (2024.eacl-long)
Copied to clipboard
| Challenge: | Uncertainty estimation is an important diagnostic tool for statistical models. |
| Approach: | They propose to adapt similarity-sensitive Shannon entropy (S3E) for NMT by incorporating a concept borrowed from theoretical ecology. |
| Outcome: | The proposed framework improves quality estimation and named entity recall, and improves translation quality. |
Towards Robust Neural Machine Translation with Iterative Scheduled Data-Switch Training (2022.coling-1)
Copied to clipboard
Zhongjian Miao, Xiang Li, Liyan Kang, Wen Zhang, Chulun Zhou, Yidong Chen, Bin Wang, Min Zhang, Jinsong Su
| Challenge: | Existing methods on robust neural machine translation (NMT) construct adversarial examples by injecting noise into authentic examples and indiscriminately exploit two types of examples. |
| Approach: | They propose an iterative scheduled data-switch training framework to mitigate this problem by injecting noise into authentic examples and indiscriminately exploiting two types of examples. |
| Outcome: | The proposed model outperforms several competitive benchmarks on four translation benchmarks. |
Improving Back-Translation with Uncertainty-based Confidence Estimation (D19-1)
Copied to clipboard
| Challenge: | Despite the success of low-resource neural machine translation, there is a data scarcity problem in many languages . large-scale, high-quality, and widecoverage bilingual corpora do not exist for most language pairs . |
| Approach: | They propose to quantify confidence of NMT models based on model uncertainty . they propose to use uncertainty-based confidence measures to improve back-translation . |
| Outcome: | The proposed model outperforms conventional statistical machine translation (SMT) on Chinese-English and English-German translation tasks. |