| Challenge: | Existing methods to improve NMT performance but there is a discrepancy between training and inference when decoding. |
| Approach: | They propose to use Scheduled Sampling to reduce the discrepancy between training and inference in NMT when decoding to mitigate the discrépancy. |
| Outcome: | The proposed methods improve translation quality over standard NMT system. |
Similar Papers
Dynamic Sentence Sampling for Efficient Training of Neural Machine Translation (P18-2)
Copied to clipboard
| Challenge: | Existing methods to train neural machine translation (NMT) use a fixed training procedure where each sentence is sampled once during each epoch. |
| Approach: | They propose to dynamically sample sentences to accelerate NMT training . a weight is assigned to each sentence based on the measured difference between training costs of two iterations. |
| Outcome: | Empirical results show that the proposed method can significantly accelerate training and improve NMT performance. |
Bridging the Gap between Training and Inference for Neural Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Neural Machine Translation generates target words sequentially while at inference it has to generate the entire sequence from scratch. |
| Approach: | They propose to use ground truth and inference to generate target words sequentially while at inference it has to generate the entire sequence from scratch. |
| Outcome: | Experiments on Chinese->English and WMT’14 English->German translation tasks show that the proposed model can achieve significant improvements on multiple datasets. |
Quality-Aware Decoding for Neural Machine Translation (2022.naacl-main)
Copied to clipboard
Patrick Fernandes, António Farinhas, Ricardo Rei, José G. C. de Souza, Perez Ogayo, Graham Neubig, Andre Martins
| Challenge: | Despite advances in machine translation quality estimation and evaluation, decoding is mostly oblivious to this. |
| Approach: | They propose to use a decoding framework that is quality-aware for neural machine translation . they compare various methods like N-best reranking and minimum Bayes risk decoding . |
| Outcome: | The proposed quality-aware decoding outperforms MAP-based decoding on four datasets and two model classes. |
Adaptive Multi-pass Decoder for Neural Machine Translation (D18-1)
Copied to clipboard
| Challenge: | End-to-end neural machine translation (NMT) has attracted increasing attention in recent years. |
| Approach: | They propose an adaptive multi-pass decoder which introduces a flexible multi- pass polishing mechanism to extend the capacity of NMT via reinforcement learning. |
| Outcome: | The proposed architecture improves Chinese-English translation with 1.55 BLEU . the proposed architecture adopts a flexible multi-pass polishing mechanism . |
Self-Paced Learning for Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have shown that the training of neural machine translation (NMT) rely on the quality of artificial schedule drawn up with the handcrafted features, e.g. sentence length or word rarity. |
| Approach: | They propose to train NMT model using a self-paced learning approach that allows it to quantify the learning confidence over training examples and flexibly govern its learning via regulating the loss in each iteration step. |
| Outcome: | The proposed model outperforms baseline models and those trained with human-designed curricula on translation quality and convergence speed. |
Scheduled Sampling Based on Decoding Steps for Neural Machine Translation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Neural machine translation models are trained to maximize the likelihood of next token given previous golden tokens as inputs, but at the inference stage, golden token is unavailable. |
| Approach: | They propose to use scheduled sampling to replace ground-truth tokens with predicted tokens to bridge the gap between training and inference. |
| Outcome: | The proposed methods outperform the Transformer baseline and vanilla scheduled sampling on three large-scale WMT tasks. |
Is MAP Decoding All You Need? The Inadequacy of the Mode in Neural Machine Translation (2020.coling-main)
Copied to clipboard
| Challenge: | Recent studies have revealed a number of pathologies of neural machine translation systems. |
| Approach: | They propose to use maximum a posteriori decoding to identify the highest-scoring translation, i.e. the mode problem, to validate the model and its training algorithm. |
| Outcome: | The proposed model reproduces the statistical data well, but the beam search strays from the statistics. |
Deconvolution-Based Global Decoding for Neural Machine Translation (C18-1)
Copied to clipboard
| Challenge: | Existing models for Neural Machine Translation (NMT) use Recurrent Neural Network (RNN) to generate translation word by word following a sequential order. |
| Approach: | They propose a Neural Machine Translation (NMT) model that decodes the sequence with the guidance of its structural prediction of the target-side context. |
| Outcome: | The proposed model is more competitive compared with the state-of-the-art methods and reduces repetition with the instruction from the target-side context for decoding. |
Understanding and Improving Sequence-to-Sequence Pretraining for Neural Machine Translation (2022.acl-long)
Copied to clipboard
| Challenge: | Existing studies on self-supervised pretraining for machine translation have focused on the jointly pretrained decoder . |
| Approach: | They propose a method to improve neural machine translation by jointly pretrained decoder . they propose two strategies to remedy the domain and objective discrepancies . |
| Outcome: | The proposed approach improves translation performance and model robustness on three language pairs. |
Fast Lexically Constrained Decoding with Dynamic Beam Allocation for Neural Machine Translation (N18-1)
Copied to clipboard
| Challenge: | Existing approaches to neural machine translation have computational complexities that are either linear or exponential in the number of constraints. |
| Approach: | They propose an algorithm for lexically constrained decoding with a complexity of O(1) in the number of constraints. |
| Outcome: | The proposed algorithm can place constraints and improve results in simulated post-editing tasks. |