Simulated multiple reference training improves low-resource machine translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing valid translations for a given sentence are limited by a single reference translation, causing data sparsity in low-resource settings. |
| Approach: | They propose a method that approximates the full space of possible translations by sampling a paraphrase of the reference sentence from a MT model and training it to predict the paraphraser’s distribution over possible tokens. |
| Outcome: | The proposed method improves in low-resource settings and is complementary to back-translation. |
Similar Papers
Leveraging Synthetic Targets for Machine Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | Using synthetic target data, training models on synthetic targets outperforms training on actual ground-truth data. |
| Approach: | They propose a recipe for training machine translation models on synthetic target data by leveraging a large pre-trained model. |
| Outcome: | The proposed model outperforms training on real-world translation datasets. |
Multilingual Denoising Pre-training for Neural Machine Translation (2020.tacl-1)
Copied to clipboard
Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvininejad, Mike Lewis, Luke Zettlemoyer
| Challenge: | Existing approaches to pre-train models focus on only English corpora, but this is not common in machine translation. |
| Approach: | They propose a sequence-to-sequence denoising auto-encoder pre-trained on monolingual corpora . they show that it produces significant performance gains across MT tasks . |
| Outcome: | The proposed model can achieve significant performance gains across a wide variety of MT tasks. |
Multi-Reference Training with Pseudo-References for Neural Translation and Text Generation (D18-1)
Copied to clipboard
| Challenge: | Neural text generation has been quite successful recently, but during training time, only one reference is considered for each example, even though there are often multiple references available. |
| Approach: | They propose an algorithm to generate exponentially many pseudo-references by compressing existing references into lattices and traversing them to generate new pseudo-References. |
| Outcome: | The proposed model significantly improves on baselines in machine translation and image captioning, and is comparable to existing models. |
Few-shot learning through contextual data augmentation (2021.eacl-main)
Copied to clipboard
| Challenge: | Various strategies have been explored to learn from a journalist's post-edits . state-of-the-art APE systems require large numbers of post- edits for training . |
| Approach: | They propose to teach a pre-trained machine translation model to translate previously unseen words accurately . they extend a data augmentation approach to create training examples with similar contexts . |
| Outcome: | The proposed model improves accuracy on the scale of one to five examples with only 1 to 5 examples. |
A Study in Improving BLEU Reference Coverage with Diverse Automatic Paraphrasing (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Using neural paraphrasing techniques, we investigate whether automatically generating additional *diverse* references can provide better coverage of the space of valid translations. |
| Approach: | They propose to use neural paraphrasing techniques to generate additional references that provide better coverage of the space of valid translations. |
| Outcome: | The proposed approach beats human paraphrases in the BLEU evaluation. |
Pre-Trained Multilingual Sequence-to-Sequence Models: A Hope for Low-Resource Language Translation? (2022.findings-acl)
Copied to clipboard
En-Shiun Lee, Sarubi Thillainathan, Shravan Nayak, Surangika Ranathunga, David Adelani, Ruisi Su, Arya McCarthy
| Challenge: | Pre-trained multilingual sequence-to-sequence models like mBART and mT5 can be used to translate low-resource languages, but their practical application is unclear. |
| Approach: | They conduct an empirical experiment in 10 languages to determine what can pre-trained multilingual sequence-to-sequence models like mBART do to translate low-resource languages? |
| Outcome: | The proposed models are robust to domain differences, but translations for unseen and typologically distant languages remain below 3.0 BLEU. |
Balancing Training for Multilingual Neural Machine Translation (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods to train multilingual machine translation models are imbalanced and heterogeneous data is wildly varying. |
| Approach: | They propose a method that automatically learns how to weight training data through a data scorer that is optimized to maximize performance on all test languages. |
| Outcome: | The proposed method outperforms baselines on two sets of languages under one-to-many and many-to-1 MT settings and offers flexible control over which languages are optimized. |
SMaLL-100: Introducing Shallow Multilingual Machine Translation Model for Low-Resource Languages (2022.emnlp-main)
Copied to clipboard
Alireza Mohammadshahi, Vassilina Nikoulina, Alexandre Berard, Caroline Brun, James Henderson, Laurent Besacier
| Challenge: | Existing models for multilingual machine translation use scaling up the number of parameters to overcome the curse of multilinguality. |
| Approach: | They propose a multilingual machine translation model that shares information between similar languages and scales up the number of parameters to overcome the curse of multilinguality. |
| Outcome: | The proposed model outperforms previous models on low-resource benchmarks while improving inference latency and memory usage. |
Phrase-Based & Neural Unsupervised Machine Translation (D18-1)
Copied to clipboard
| Challenge: | Recent advances in machine translation have reported near human-level performance on several languages, yet their effectiveness strongly relies on the availability of large amounts of parallel sentences. |
| Approach: | They propose two models that leverage a careful initialization of the parameters and denoising effect of language models. |
| Outcome: | The proposed models outperform the current methods on English-French and German-English benchmarks while being simpler and having fewer hyper-parameters. |
Breaking Down Multilingual Machine Translation (2022.findings-acl)
Copied to clipboard
| Challenge: | Multilingual training is an essential ingredient in machine translation systems . but it has different effects in different multilingual settings, such as many-to-one, one-tomany and many- to-many learning . |
| Approach: | They compare multilingual training settings with encoders and decoders initialized by multilingual learning . they find important attention heads for each language pair and compare their correlations during inference . |
| Outcome: | The proposed models outperform the best models for high-resource languages and one-to-many models for low-resourced languages. |