DEEP: DEnoising Entity Pre-training for Neural Machine Translation (2022.acl-long)
Copied to clipboard
| Challenge: | Earlier named entity translation methods focus on phonetic transliteration, which ignores the sentence context for translation. |
| Approach: | They propose a DEnoising Entity Pre-training method that leverages monolingual data and a knowledge base to improve named entity translation accuracy within sentences. |
| Outcome: | The proposed method improves on three language pairs and denoising auto-encoding baselines. |
Similar Papers
Multilingual Denoising Pre-training for Neural Machine Translation (2020.tacl-1)
Copied to clipboard
Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvininejad, Mike Lewis, Luke Zettlemoyer
| Challenge: | Existing approaches to pre-train models focus on only English corpora, but this is not common in machine translation. |
| Approach: | They propose a sequence-to-sequence denoising auto-encoder pre-trained on monolingual corpora . they show that it produces significant performance gains across MT tasks . |
| Outcome: | The proposed model can achieve significant performance gains across a wide variety of MT tasks. |
Synthetic Pre-Training Tasks for Neural Machine Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | toxicity and bias can be addressed by pre-training with synthetic resources . BLEU scores are used to compare methods with real-world data . |
| Approach: | They propose several ways to generate obfuscated data from large parallel corpus and concatenating phrase pairs from small word-aligned corpus with synthetic parallel data without real human language corpora. |
| Outcome: | The proposed methods can be used to generate obfuscated data or synthetic parallel data without real human language corpora even with high levels of oblication. |
Phrase-Based & Neural Unsupervised Machine Translation (D18-1)
Copied to clipboard
| Challenge: | Recent advances in machine translation have reported near human-level performance on several languages, yet their effectiveness strongly relies on the availability of large amounts of parallel sentences. |
| Approach: | They propose two models that leverage a careful initialization of the parameters and denoising effect of language models. |
| Outcome: | The proposed models outperform the current methods on English-French and German-English benchmarks while being simpler and having fewer hyper-parameters. |
Neural Machine Translation Incorporating Named Entity (C18-1)
Copied to clipboard
| Challenge: | Conventional NMT models have difficulty translating words with multiple meanings because of the high ambiguity. |
| Approach: | They propose a neural machine translation model that incorporates named entity (NE) tags of source-language sentences to reduce the difficulty in translating multiple meanings. |
| Outcome: | The proposed model achieves 3.11 point improvement in bilingual evaluation understudy (BLEU) on English-to-Japanese translation task with the ASPEC, and English- to-Bulgarian and English to-Romanian translation tasks with the Europarl corpus. |
Shallow-to-Deep Training for Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results show that deep training is 1:4 faster than training from scratch. |
| Approach: | They propose a shallow-to-deep training method that learns deep models by stacking shallow models. |
| Outcome: | The proposed method is 1:4 faster than training from scratch and achieves BLEU scores of 30:33 and 43:29 on two translation tasks. |
Leveraging Synthetic Targets for Machine Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | Using synthetic target data, training models on synthetic targets outperforms training on actual ground-truth data. |
| Approach: | They propose a recipe for training machine translation models on synthetic target data by leveraging a large pre-trained model. |
| Outcome: | The proposed model outperforms training on real-world translation datasets. |
Addressing Entity Translation Problem via Translation Difficulty and Context Diversity (2024.findings-acl)
Copied to clipboard
| Challenge: | Neural machine translation systems often produce inadequate translations for named entities. |
| Approach: | They propose a data augmentation strategy to enhance the accuracy of named entity translation by retraining the target named entity pair. |
| Outcome: | The proposed method improves translation accuracy across test sets and terminology tests. |
Breaking the Corpus Bottleneck for Context-Aware Neural Machine Translation with Cross-Task Pre-training (2021.acl-long)
Copied to clipboard
| Challenge: | Context-aware neural machine translation (NMT) remains challenging due to the lack of large-scale document-level parallel corpora. |
| Approach: | They propose to use large-scale parallel datasets and source-side monolingual documents to improve context-aware neural machine translation. |
| Outcome: | The proposed model can be used to translate both sentences and documents on four translation tasks. |
Human or Neural Translation? (2020.coling-main)
Copied to clipboard
Shivendra Bhardwaj, David Alfonso Hermelo, Phillippe Langlais, Gabriel Bernier-Colborne, Cyril Goutte, Michel Simard
| Challenge: | a recent study shows that deep neural models have improved machine translation . identifying machine translation is still feasible, but is not yet known. |
| Approach: | They train and apply deep neural models to distinguish between human and machine translations . they use a monolingual and bilingual task to train and train 18 classifiers based on their results . |
| Outcome: | The proposed model improves the ability to distinguish between human and machine translations at the sentence level. |
Bridging the Gap between Training and Inference for Neural Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Neural Machine Translation generates target words sequentially while at inference it has to generate the entire sequence from scratch. |
| Approach: | They propose to use ground truth and inference to generate target words sequentially while at inference it has to generate the entire sequence from scratch. |
| Outcome: | Experiments on Chinese->English and WMT’14 English->German translation tasks show that the proposed model can achieve significant improvements on multiple datasets. |