Code-Switching with Word Senses for Pretraining in Neural Machine Translation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Lexical ambiguity is a significant and pervasive challenge in Neural Machine Translation (NMT) many state-of-the-art (SOTA) NMT systems struggle to handle polysemous words . |
| Approach: | They propose an end-to-end approach for pretraining multilingual NMT models leveraging word sense-specific information from Knowledge Bases. |
| Outcome: | The proposed approach improves translation quality and scales to various data and resource-strapped scenarios. |
Similar Papers
Exploring Enhanced Code-Switched Noising for Pretraining in Neural Machine Translation (2023.findings-eacl)
Copied to clipboard
| Challenge: | Multilingual pretraining approaches to denoise synthetic code-switched data have shown that they generate the noise using non-contextual, one-to-one word translations obtained from lexicons. |
| Approach: | They propose an approach where contextual, many-to-many word translations are generated using a ‘base’ NMT model. |
| Outcome: | The proposed approach improves on 3 different language families and shows that small models can perform better than massive models like mBART50 and mRASP2 . |
CSP:Code-Switching Pre-training for Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to train neural machine translation models are data-hungry and low-resource . et al., 2018; Radford e.t., 2019; Yang ee.,2019) proposes a new pre-training method for NMT . |
| Approach: | They propose a new pre-training method which randomly replaces some words in the input sentence with their translation words in target language. |
| Outcome: | The proposed method improves on unsupervised and supervised NMT models by making full use of monolingual corpora. |
Reducing Disambiguation Biases in NMT by Leveraging Explicit Word Sense Information (2022.naacl-main)
Copied to clipboard
| Challenge: | Recent studies show that Neural Machine Translation models struggle to disambiguate polysemous words without lapsing into their most frequent senses. |
| Approach: | They propose a way to automatically create high-precision sense-annotated parallel corpora . they then propose 'fine-tuning' strategies to exploit these sense annotations during training . |
| Outcome: | The proposed approach achieves higher BLEU scores than its vanilla counterpart in 3 language pairs. |
Bridging the Gap between Training and Inference for Neural Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Neural Machine Translation generates target words sequentially while at inference it has to generate the entire sequence from scratch. |
| Approach: | They propose to use ground truth and inference to generate target words sequentially while at inference it has to generate the entire sequence from scratch. |
| Outcome: | Experiments on Chinese->English and WMT’14 English->German translation tasks show that the proposed model can achieve significant improvements on multiple datasets. |
Handling Homographs in Neural Machine Translation (N18-1)
Copied to clipboard
| Challenge: | Existing methods for MT have problems with translating homographs, as it is difficult to select the correct translation based on the context. |
| Approach: | They propose to model the context of the input word with context-aware word embeddings that help to differentiate the word sense before feeding it into the encoder. |
| Outcome: | The proposed models improve translation accuracy and BLEU score on three language pairs. |
Encoders Help You Disambiguate Word Senses in Neural Machine Translation (D19-1)
Copied to clipboard
| Challenge: | Neural machine translation models can perform word sense disambiguation (WSD) however, it is unclear which component dominates the process of disambiguating words. |
| Approach: | They evaluate hidden states and investigate distributions of self-attention in NMT encoders and decoders to disambiguate word senses. |
| Outcome: | The proposed model outperforms encoder hidden states on large datasets . the model outpersforms decoders on large data sets . |
Understanding and Improving Sequence-to-Sequence Pretraining for Neural Machine Translation (2022.acl-long)
Copied to clipboard
| Challenge: | Existing studies on self-supervised pretraining for machine translation have focused on the jointly pretrained decoder . |
| Approach: | They propose a method to improve neural machine translation by jointly pretrained decoder . they propose two strategies to remedy the domain and objective discrepancies . |
| Outcome: | The proposed approach improves translation performance and model robustness on three language pairs. |
Pivot-based Transfer Learning for Neural Machine Translation between Non-English Languages (D19-1)
Copied to clipboard
| Challenge: | Using parallel corpora, we train a single, direct NMT model for non-English language pairs. |
| Approach: | They propose three ways to increase the relation among source, pivot, and target languages in pre-training . they use additional adapter component to smoothly connect pre-trained encoder and decoder . |
| Outcome: | The proposed methods outperform multilingual models up to +2.6% BLEU in WMT 2019 French-German and German-Czech tasks. |
Code-Switching for Enhancing NMT with Pre-Specified Translation (N19-1)
Copied to clipboard
| Challenge: | Existing methods to constrain NMT use placeholder tags for lexicon words and hard constraints during decoding. |
| Approach: | They propose to use placeholder tags to replace lexicon words with target translations . they use a data augmentation method to make code-switched training data . |
| Outcome: | The proposed method improves translation quality without hurting unconstrained words. |
Breaking the Corpus Bottleneck for Context-Aware Neural Machine Translation with Cross-Task Pre-training (2021.acl-long)
Copied to clipboard
| Challenge: | Context-aware neural machine translation (NMT) remains challenging due to the lack of large-scale document-level parallel corpora. |
| Approach: | They propose to use large-scale parallel datasets and source-side monolingual documents to improve context-aware neural machine translation. |
| Outcome: | The proposed model can be used to translate both sentences and documents on four translation tasks. |