Long Warm-up and Self-Training: Training Strategies of NICT-2 NMT System at WAT-2019 (D19-52)
Copied to clipboard
| Challenge: | NICT-2 neural machine translation system was presented at the 6th Workshop on Asian Translation (WAT-2019) |
| Approach: | They describe a NICT-2 neural machine translation system at the 6th Workshop on Asian Translation . they employ a long warm-up strategy and a self-training strategy that uses multiple back-translations generated by sampling to improve the translation quality. |
| Outcome: | The proposed system improves translation quality and learning rate by using the long warm-up and self-training strategies. |
Similar Papers
English-Myanmar Supervised and Unsupervised NMT: NICT’s Machine Translation Systems at WAT-2019 (D19-52)
Copied to clipboard
| Challenge: | NICT participated in the 6th Workshop on Asian Translation (WAT-2019) shared translation task, specifically Myanmar (My) - English task in both translation directions. |
| Approach: | They present the participation of the NICT in the 6th Workshop on Asian Translation (WAT-2019) shared translation task, specifically Myanmar (Burmese) - English task in both translation directions. |
| Outcome: | The proposed systems perform the third in English-to-Myanmar and the second in Myanmar-to English according to BLEU score. |
Our Neural Machine Translation Systems for WAT 2019 (D19-52)
Copied to clipboard
| Challenge: | In the last five years, statistical machine translation is gradually fading out in favor of neural machine translation. |
| Approach: | They describe a novel Neural Machine Translation (NMT) system for the WAT 2019 translation tasks they focus on. |
| Outcome: | The proposed system improves translation accuracy while replacing absolute position representations with relative positions. |
Pre-training Methods for Neural Machine Translation (2021.acl-tutorials)
Copied to clipboard
| Challenge: | This tutorial provides a comprehensive guide to make the most of pre-training for neural machine translation. |
| Approach: | This tutorial provides a comprehensive guide to make the most of pre-training for neural machine translation. |
| Outcome: | This tutorial explains how to make the most of pre-training for neural machine translation. |
Self-Paced Learning for Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have shown that the training of neural machine translation (NMT) rely on the quality of artificial schedule drawn up with the handcrafted features, e.g. sentence length or word rarity. |
| Approach: | They propose to train NMT model using a self-paced learning approach that allows it to quantify the learning confidence over training examples and flexibly govern its learning via regulating the loss in each iteration step. |
| Outcome: | The proposed model outperforms baseline models and those trained with human-designed curricula on translation quality and convergence speed. |
Self-Supervised Neural Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) methods relied on the availability of high-quality parallel corpora. |
| Approach: | They propose a method where an emergent NMT system is used for selecting training data and learning internal NMT representations. |
| Outcome: | The proposed method achieves BLEU scores of 29.21 (en2fr) and 27.36 (fr2en) on newstest2014 using English and French Wikipedia data for training. |
Active Learning Approaches to Enhancing Neural Machine Translation (2020.findings-emnlp)
Copied to clipboard
| Challenge: | a limited human translation budget is required to train neural machine translation models. |
| Approach: | They propose to integrate active learning into neural machine translation techniques . they propose a word frequency based acquisition function and an uncertainty based method . |
| Outcome: | The proposed method outperforms other acquisition functions on a limited human translation budget. |
Learning When to Concentrate or Divert Attention: Self-Adaptive Attention Temperature for Neural Machine Translation (D18-1)
Copied to clipboard
| Challenge: | Neural Machine Translation models treat decoding at each time step equally with the same matrix . conventional methods treat decoder outputs at all time steps with the identical weight matrix causing inaccuracy . |
| Approach: | They propose a model with a mechanism to control the softness of attention by means of an attention temperature. |
| Outcome: | The proposed model outperforms baseline models on Chinese-English and English-Vietnamese translations. |
LTRC-MT Simple & Effective Hindi-English Neural Machine Translation Systems at WAT 2019 (D19-52)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) is a promising approach for low resource languages. |
| Approach: | They propose to use both Recurrent Neural Networks & Transformer architectures to train NMT models. |
| Outcome: | The proposed model outperforms Statistical Machine Translation (SMT) techniques on a low resource Hindi-English language pair. |
Data Selection Curriculum for Neural Machine Translation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Neural Machine Translation models are typically trained on heterogeneous data that are concatenated and randomly shuffled. |
| Approach: | They propose a two-stage curriculum training framework where a NMT model is fine-tuned on subsets of data, selected by deterministic scoring and online scoring. |
| Outcome: | The proposed framework improves on six language pairs comprising low- and high-resource languages and shows up to +2.2 BLEU improvement and faster convergence. |
Supervised neural machine translation based on data augmentation and improved training & inference process (D19-52)
Copied to clipboard
| Challenge: | This paper describes the neural machine translation systems for the shared translation tasks of WAT 2019 . |
| Approach: | They propose a model for translation tasks of WAT 2019 that employs a Transformer model as the baseline and a deep layer model to improve translation quality. |
| Outcome: | The proposed methods can improve translation quality over traditional statistical machine translation (SMT) The proposed models can improve the translation quality of Japanese-English and Japanese-Chinese corpus. |