Beyond BLEU:Training Neural Machine Translation with Semantic Similarity (P19-1)
Copied to clipboard
| Challenge: | Recent work has shown that optimizing neural machine translation systems to directly improve evaluation metrics such as BLEU can improve final translation accuracy. |
| Approach: | They propose a reward function that assigns partial credit to BLEU and provides more diversity in scores than BLUE. |
| Outcome: | The proposed reward function improves translation accuracy, semantic similarity, and human evaluation on four languages trans-lated to English and the optimization procedure converges faster. |
Similar Papers
Are BLEU and Meaning Representation in Opposition? (P18-1)
Copied to clipboard
| Challenge: | Empirical evaluation suggests that the better the translation quality, the worse the learned sentence representations serve in a wide range of classification and similarity tasks. |
| Approach: | They propose several variations of the attentive NMT architecture to bring this meeting point back . they propose to use a structured fixed-size representation of the input to produce static representations of input sentences. |
| Outcome: | The proposed architecture improves translation quality and performance in a range of tasks. |
Using Context in Neural Machine Translation Training Objectives (2020.acl-main)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) training is based on document-level metrics, not sentence-level BLEU. |
| Approach: | They propose to merge document-level metrics with batch-level documents to improve NMT training. |
| Outcome: | The proposed training is more robust for document-level metrics than sequence MRT and maximum-likelihood training. |
Energy-Based Reranking: Improving Neural Machine Translation Using Energy-Based Models (2021.acl-long)
Copied to clipboard
Sumanta Bhattacharyya, Amirmohammad Rooshenas, Subhajit Naskar, Simeng Sun, Mohit Iyyer, Andrew McCallum
| Challenge: | Autoregressive neural machine translation (NMT) uses a tractable likelihood computation and efficient sampling. |
| Approach: | They propose to use an energy-based model to mimic the behavior of the task measure and use it to train an energy based re-ranking algorithm. |
| Outcome: | The proposed model improves on the samples drawn from the NMT with a higher BLEU score than the experimental model and the energy-based re-ranking algorithm. |
Bridging the Gap between Training and Inference for Neural Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Neural Machine Translation generates target words sequentially while at inference it has to generate the entire sequence from scratch. |
| Approach: | They propose to use ground truth and inference to generate target words sequentially while at inference it has to generate the entire sequence from scratch. |
| Outcome: | Experiments on Chinese->English and WMT’14 English->German translation tasks show that the proposed model can achieve significant improvements on multiple datasets. |
When and Why Are Pre-Trained Word Embeddings Useful for Neural Machine Translation? (N18-2)
Copied to clipboard
| Challenge: | Pre-trained word embeddings have proven to be invaluable for improving performance in natural language analysis tasks where large-scale parallel corpora cannot be obtained. |
| Approach: | They perform five sets of experiments to analyze when pre-trained word embeddings can be useful in NMT tasks. |
| Outcome: | The embeddings provide gains of up to 20 BLEU points in the most favorable setting. |
Reducing Disambiguation Biases in NMT by Leveraging Explicit Word Sense Information (2022.naacl-main)
Copied to clipboard
| Challenge: | Recent studies show that Neural Machine Translation models struggle to disambiguate polysemous words without lapsing into their most frequent senses. |
| Approach: | They propose a way to automatically create high-precision sense-annotated parallel corpora . they then propose 'fine-tuning' strategies to exploit these sense annotations during training . |
| Outcome: | The proposed approach achieves higher BLEU scores than its vanilla counterpart in 3 language pairs. |
Using Semantic Role Labeling to Improve Neural Machine Translation (2022.lrec-1)
Copied to clipboard
| Challenge: | despite progress in machine translation, some form of language understanding may be desirable . current systems rely on pattern recognition, but some form may be useful . |
| Approach: | They use semantic role labeling to annotate a standard parallel corpus with semantic roles . they then train a neural machine translation system using the annotated corpus and original unannotated text . |
| Outcome: | The proposed system improves BLEU scores for English, French, German, Greek and Spanish. |
A Simple and Effective Approach to Coverage-Aware Neural Machine Translation (P18-2)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) models are used to solve translation problems using long-term models. |
| Approach: | They propose a method to seek a better balance between model confidence and length preference for Neural Machine Translation. |
| Outcome: | The proposed model improves on Chinese-English and English-German translation tasks. |
When do Contrastive Word Alignments Improve Many-to-many Neural Machine Translation? (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods to improve pre-training for many-to-many neural machine translation use manual cleaning of bilingual dictionaries, which are unavailable for most language pairs. |
| Approach: | They propose a word-level contrastive objective to leverage word alignments for many-to-many neural machine translation (NMT) Empirical results show that this leads to 0.8 BLEU gains for several language pairs. |
| Outcome: | Empirical results show that the proposed objective leads to 0.8 BLEU gains for several language pairs. |
It’s Easier to Translate out of English than into it: Measuring Neural Translation Difficulty by Cross-Mutual Information (2020.acl-main)
Copied to clipboard
| Challenge: | Current state-of-the-art MT systems are based on neural networks, but it is unclear whether all translation directions are equally easy (or hard) to model for NMT. |
| Approach: | They propose an asymmetric information-theoretic metric of machine translation difficulty that exploits the probabilistic nature of most neural machine translation models. |
| Outcome: | The proposed metric allows us to better evaluate the difficulty of translating text into the target language while controlling for the difficulty independent of the translation task. |