Better Neural Machine Translation by Extracting Linguistic Information from BERT (2021.eacl-main)
Copied to clipboard
| Challenge: | Experimental results show that incorporating linguistic information into neural machine translation models is no more difficult to train than conventional Transformer-based NMT. |
| Approach: | They propose to extract linguistic information from contextual word embeddings instead of point estimates to augment NMT models. |
| Outcome: | The proposed method generalizes better in a variety of training contexts and is no more difficult to train than conventional Transformer-based NMT. |
Similar Papers
BERTology for Machine Translation: What BERT Knows about Linguistic Difficulties for Translation (2022.lrec-1)
Copied to clipboard
| Challenge: | Pre-trained transformer-based models have shown excellent performance in most benchmark tests, but lack a good understanding of the linguistic knowledge of BERT in Neural Machine Translation (NMT). |
| Approach: | They propose to use QE models to analyze BERT's syntactic dependencies and their impact on machine translation quality. |
| Outcome: | The proposed model is able to model with self-attention in the pre-training phase, which improves generalization ability. |
Linguistically Informed Hindi-English Neural Machine Translation (2020.lrec-1)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) is a promising approach to machine translation . lack of parallel training data for Hindi-English is limiting . |
| Approach: | They propose to incorporate linguistic knowledge encoded by Hindi phenomena into a Transformer model to improve the translation performance. |
| Outcome: | The proposed model incorporates linguistic features to improve the translation performance. |
Improving Low-Resource NMT through Relevance Based Linguistic Features Incorporation (2020.coling-main)
Copied to clipboard
| Challenge: | Existing studies on incorporating arbitrary syntactic information into neural machine translation (NMT) are lacking. |
| Approach: | They propose to integrate linguistic knowledge at different levels into neural machine translation framework to improve translation quality for language pairs with extremely limited data. |
| Outcome: | The proposed methods improve translation quality for all tasks by 3.09 BLEU points . the proposed methods are based on two different approaches . |
On the use of BERT for Neural Machine Translation (D19-56)
Copied to clipboard
| Challenge: | Existing studies on using pretrained language models for supervised NMT have not been successful. |
| Approach: | They propose to integrate BERT pretrained models with supervised NMT models by using monolingual data. |
| Outcome: | The proposed models improve translation quality in English-German, English-Russian and IWSLT14 datasets. |
A Survey of Domain Adaptation for Neural Machine Translation (C18-1)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) is a deep learning based approach for machine translation. |
| Approach: | They propose to use a deep learning approach to train machine translation in scenarios where large-scale parallel corpora are available. |
| Outcome: | The proposed approach yields the state-of-the-art translation performance in resource rich scenarios. |
Multilingual Neural Machine Translation (2020.coling-tutorials)
Copied to clipboard
| Challenge: | In this tutorial, we will cover the latest advances in NMT to enhance low-resource translation. |
| Approach: | They will cover the latest advances in NMT approaches that leverage multilingualism . they will focus on topics such as language divergence, transfer learning and pivoting . |
| Outcome: | This tutorial will cover the latest advances in NMT to enhance low-resource translation models. |
Bridging the Gap between Training and Inference for Neural Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Neural Machine Translation generates target words sequentially while at inference it has to generate the entire sequence from scratch. |
| Approach: | They propose to use ground truth and inference to generate target words sequentially while at inference it has to generate the entire sequence from scratch. |
| Outcome: | Experiments on Chinese->English and WMT’14 English->German translation tasks show that the proposed model can achieve significant improvements on multiple datasets. |
Examining the Tip of the Iceberg: A Data Set for Idiom Translation (L18-1)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) has been widely used in recent years with significant improvements for many language pairs. |
| Approach: | They propose to use a large-scale data set to evaluate idiom translation in GermanEnglish. |
| Outcome: | The proposed dataset is used to perform preliminary NMT experiments on idiom translation in GermanEnglish. |
A Comparison between Pre-training and Large-scale Back-translation for Neural Machine Translation (2021.findings-acl)
Copied to clipboard
| Challenge: | BERT is a promising technique to improve NMT, but how it outperforms standard NMT is understudied. |
| Approach: | We compare MT engines trained with pre-trained BERT and back-translation with incrementally larger amounts of data. |
| Outcome: | The proposed technique outperforms standard NMT models on morphology and syntax. |
Categorizing Semantic Representations for Neural Machine Translation (2022.coling-1)
Copied to clipboard
| Challenge: | Modern neural machine translation models suffer limitation in compositional generalization, resulting in weakened translation performance on unseen compounds. |
| Approach: | They propose to introduce categorization to the contextualized representations to improve generalization by reducing sparsity and overfitting. |
| Outcome: | The proposed method reduces compositional generalization error rates by 24% on a dedicated MT dataset. |