| Challenge: | Existing weighted sum models (WSMs) take inputs and generate one output, but they are independent of each other and are fixed for all inputs. |
| Approach: | They propose adaptive weighting for WSMs to control the contribution of each input and output state. |
| Outcome: | The proposed weighting improves translation accuracy by 1.49 and 0.92 BLEU points on Chinese-to-English translation and English-to German translation tasks. |
Similar Papers
Domain Adaptive Inference for Neural Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Neural Machine Translation models are effective when trained on broad domains with large datasets, such as news translation. |
| Approach: | They propose a novel approach for adaptive ensemble weighting for Neural Machine Translation by extending Bayesian Interpolation with source information. |
| Outcome: | The proposed approach improves performance on Spanish-English and English-German tasks without the need for the domain label. |
Unsupervised Neural Machine Translation with Weight Sharing (P18-1)
Copied to clipboard
| Challenge: | Unsupervised neural machine translation (NMT) is a new approach for machine translation . the model uses only one shared encoder to map pairs of sentences from different languages to a shared-latent space . |
| Approach: | They propose an unsupervised approach which trains the model without labeling data . they propose two independent encoders but share some partial weights to extract high-level representations of input sentences. |
| Outcome: | The proposed approach achieves significant improvements on English-German, English-French and Chinese-to-English translation tasks. |
A Simple and Effective Approach to Coverage-Aware Neural Machine Translation (P18-2)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) models are used to solve translation problems using long-term models. |
| Approach: | They propose a method to seek a better balance between model confidence and length preference for Neural Machine Translation. |
| Outcome: | The proposed model improves on Chinese-English and English-German translation tasks. |
Sentence Weighting for Neural Machine Translation Domain Adaptation (C18-1)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) has achieved satisfactory performance on many language pairs with various advantages over statistical machine translation. |
| Approach: | They propose a new sentence weighting method for the domain adaptation of neural machine translation that uses a domain similarity metric to evaluate the relevance of sentences to the target domain. |
| Outcome: | The proposed method achieves significant improvement over baselines on Chinese-English TED task and synthetic training task with only synthetic training parallel data. |
Dynamic Data Selection and Weighting for Iterative Back-Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results demonstrate that our methods achieve improvements of up to 1.8 BLEU points over competitive baselines. |
| Approach: | They propose a data selection and weighting strategy to iterate back-translation models and apply it to it . they use a target language to back-transcribe monolingual data, which is of high quality and reflect the target domain. |
| Outcome: | The proposed approach achieves 1.8 BLEU points over baselines on domain adaptation, low-resource, and high-resourced MT settings and on two language pairs. |
Dynamic Sentence Sampling for Efficient Training of Neural Machine Translation (P18-2)
Copied to clipboard
| Challenge: | Existing methods to train neural machine translation (NMT) use a fixed training procedure where each sentence is sampled once during each epoch. |
| Approach: | They propose to dynamically sample sentences to accelerate NMT training . a weight is assigned to each sentence based on the measured difference between training costs of two iterations. |
| Outcome: | Empirical results show that the proposed method can significantly accelerate training and improve NMT performance. |
On Compositional Generalization of Neural Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Modern neural machine translation models have shown competitive performance in benchmarks such as WMT, but there are significant issues such as robustness, domain generalization, etc. |
| Approach: | They propose a benchmark dataset for NMT models from the perspective of compositional generalization and quantitatively analyze the results. |
| Outcome: | The proposed model performs well under traditional metrics, but is low in out-of-domain and low-resource conditions. |
A Survey of Domain Adaptation for Neural Machine Translation (C18-1)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) is a deep learning based approach for machine translation. |
| Approach: | They propose to use a deep learning approach to train machine translation in scenarios where large-scale parallel corpora are available. |
| Outcome: | The proposed approach yields the state-of-the-art translation performance in resource rich scenarios. |
Simple, Scalable Adaptation for Neural Machine Translation (D19-1)
Copied to clipboard
| Challenge: | Recent advances in deep learning have led to significantly improved quality on Neural Machine Translation (NMT) however, performance on out-of-domain data or low resource languages remains poor. |
| Approach: | They propose a simple yet efficient approach for adapting pre-trained models to multiple tasks simultaneously. |
| Outcome: | The proposed approach is on par with full fine-tuning on domain adaptation and massively multilingual NMT on a massively multilingual dataset. |
Attention Weights in Transformer NMT Fail Aligning Words Between Sequences but Largely Explain Model Predictions (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Using attention weights, we show that NMT models make alignment errors by relying on uninformative tokens from the source sequence. |
| Approach: | They propose to use attention weights to regulate alignment errors in NMT models . they propose methods that largely reduce the word alignment error rate compared to standard induced alignments from attention weighted tokens. |
| Outcome: | The proposed methods reduce the word alignment error rate compared to standard induced alignments from attention weights. |