| Challenge: | Experimental results show that the proposed method outperforms strong baselines in terms of BLEU score (+1.17/+1.56) and training speedup (2.22x/3.33x). |
| Approach: | They propose a norm-based curriculum learning method that measures difficulty, competence and weight of a sentence in a word embedding. |
| Outcome: | The proposed method outperforms baselines in terms of BLEU score (+1.17/+1.56) and training speedup (2.22x/3.33x). |
Similar Papers
Token-wise Curriculum Learning for Neural Machine Translation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing curriculum learning approaches to Neural Machine Translation (NMT) require sampling sufficient amounts of “easy” samples from training data at the early stage of training. |
| Approach: | They propose a token-wise curriculum learning approach that creates sufficient amounts of easy samples from training data. |
| Outcome: | The proposed approach outperforms baselines on five language pairs on low-resource languages. |
Reinforcement Learning based Curriculum Optimization for Neural Machine Translation (N19-1)
Copied to clipboard
| Challenge: | a heterogeneous training dataset can vary in characteristics such as domain, translation quality, and degree of difficulty. |
| Approach: | They propose to use reinforcement learning to learn an optimal curriculum for NMT training . they find it can beat uniform baselines and hand-designed, state-of-the-art curricula . |
| Outcome: | The proposed approach beats baselines and hand-designed curricula on English-to-French datasets. |
Self-Paced Learning for Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have shown that the training of neural machine translation (NMT) rely on the quality of artificial schedule drawn up with the handcrafted features, e.g. sentence length or word rarity. |
| Approach: | They propose to train NMT model using a self-paced learning approach that allows it to quantify the learning confidence over training examples and flexibly govern its learning via regulating the loss in each iteration step. |
| Outcome: | The proposed model outperforms baseline models and those trained with human-designed curricula on translation quality and convergence speed. |
Competence-based Curriculum Learning for Neural Machine Translation (N19-1)
Copied to clipboard
| Challenge: | Existing NMT systems require specialized heuristics and large batch sizes. |
| Approach: | They propose a curriculum learning framework for NMT that reduces training time and costs . framework consists of a principled way of deciding which training samples are shown to the model . |
| Outcome: | The proposed framework can reduce training time and improve performance of recurrent neural network models and Transformers. |
Uncertainty-Aware Curriculum Learning for Neural Machine Translation (2020.acl-main)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) has proven to be facilitated by curriculum learning which presents examples in an easy-to-hard order at different training stages. |
| Approach: | They propose to use an uncertainty-aware curriculum learning approach to assess data difficulty and model competence to provide examples in an easy-to-hard order at different training stages. |
| Outcome: | The proposed approach outperforms baseline and related methods on translation quality and convergence speed. |
Data Selection Curriculum for Neural Machine Translation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Neural Machine Translation models are typically trained on heterogeneous data that are concatenated and randomly shuffled. |
| Approach: | They propose a two-stage curriculum training framework where a NMT model is fine-tuned on subsets of data, selected by deterministic scoring and online scoring. |
| Outcome: | The proposed framework improves on six language pairs comprising low- and high-resource languages and shows up to +2.2 BLEU improvement and faster convergence. |
Dynamic Sentence Sampling for Efficient Training of Neural Machine Translation (P18-2)
Copied to clipboard
| Challenge: | Existing methods to train neural machine translation (NMT) use a fixed training procedure where each sentence is sampled once during each epoch. |
| Approach: | They propose to dynamically sample sentences to accelerate NMT training . a weight is assigned to each sentence based on the measured difference between training costs of two iterations. |
| Outcome: | Empirical results show that the proposed method can significantly accelerate training and improve NMT performance. |
Revisiting Low-Resource Neural Machine Translation: A Case Study (P19-1)
Copied to clipboard
| Challenge: | Recent research has shown that neural machine translation models are highly data-inefficient and underperform phrase-based statistical machine translation (PBSMT) in low-resource settings. |
| Approach: | They propose to use auxiliary data to train low-resource neural machine translation systems without auxiliary monolingual or multilingual data. |
| Outcome: | The proposed methods outperform PBSMT and other statistical machine translation models in Korean–English with minimal data. |
A Simple and Effective Approach to Coverage-Aware Neural Machine Translation (P18-2)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) models are used to solve translation problems using long-term models. |
| Approach: | They propose a method to seek a better balance between model confidence and length preference for Neural Machine Translation. |
| Outcome: | The proposed model improves on Chinese-English and English-German translation tasks. |
Language Modeling, Lexical Translation, Reordering: The Training Process of NMT through the Lens of Classical SMT (2021.emnlp-main)
Copied to clipboard
| Challenge: | Statistical MT decomposes the translation task into distinct components that are learned separately. |
| Approach: | They show that neural machine translation models acquire different competences over the course of training . previous work shows how to improve some of the competences in NMT by using lexical translation probabilities, phrase memories, alignment information. |
| Outcome: | The proposed model improves translation quality and word-by-word translation, while learning complex reordering patterns. |