Beyond Error Propagation in Neural Machine Translation: Characteristics of Language Also Matter (D18-1)
Copied to clipboard
| Challenge: | Neural machine translation suffers from exposure bias and error propagation problem. |
| Approach: | They conduct a series of analyses to deeply understand the accuracy drop problem . they find that the left part of the translated sentence is often better than its right part . |
| Outcome: | The results show that the left part of the translated sentence is often better than its right part in left-to-right decoding models. |
Similar Papers
On the Language Coverage Bias for Neural Machine Translation (2021.findings-acl)
Copied to clipboard
| Challenge: | Language coverage bias is important for neural machine translation because of the target-original training data. |
| Approach: | They propose two approaches to alleviate the language coverage bias problem by explicitly distinguishing between the source-and target-original training data. |
| Outcome: | The proposed methods improve translation tasks on both back-and forward-translation and their tagged variants. |
Revisiting Negation in Neural Machine Translation (2021.tacl-1)
Copied to clipboard
| Challenge: | Negation is an important linguistic phenomenon in machine translation, as errors in translating negation may change the meaning of source sentences completely. |
| Approach: | They evaluate the translation of negation in English–German (EN–DE) and English– Chinese (EN-ZH) . they find that NMT models can distinguish negation and non-negation tokens very well and encode a lot of information about negation . |
| Outcome: | The accuracy of manual evaluation in ENDE, DEEN, ENZH, and ZHEN is 95.7%, 94.8%, 93.4%, and 91.7% respectively. |
A Natural Bias for Language Generation Models (2023.acl-short)
Copied to clipboard
| Challenge: | a standard probabilistic model for language generation has likely not yet learnt many semantic or syntactic rules of natural language, making it difficult to estimate the probability distribution over next tokens. |
| Approach: | They propose to initialise bias terms in a model's final linear layer with the log-unigram distribution and use it to output the unigram frequency statistics as prior knowledge. |
| Outcome: | The proposed method improves learning efficiency and improves overall performance. |
The Importance of Generation Order in Language Modeling (D18-1)
Copied to clipboard
| Challenge: | Neural language models are universally autoregressive, generating sentences one token at a time from left to right. |
| Approach: | They propose a two-pass language model that generates partially-filled sentences and fills in missing tokens. |
| Outcome: | The proposed model produces partially-filled sentences and fills in missing tokens. |
The Lazy Encoder: A Fine-Grained Analysis of the Role of Morphology in Neural Machine Translation (D18-1)
Copied to clipboard
| Challenge: | Neural sequence-to-sequence models have proven effective for machine translation, but at the expense of interpretability. |
| Approach: | They analyze how morphological features are captured at different levels of the NMT encoder while varying the target language. |
| Outcome: | The proposed model is not interpretable, but only captures morphological features in context and only to the extent they are directly transferable to the target words. |
A Stochastic Decoder for Neural Machine Translation (P18-1)
Copied to clipboard
| Challenge: | Neural machine translation models do not account for local lexical and syntactic variation in parallel corpora. |
| Approach: | They propose a deep generative model of machine translation which incorporates a chain of latent variables to account for local lexical and syntactic variation in parallel corpora. |
| Outcome: | The proposed model consistently improves over strong baselines on several different language pairs. |
Bridging the Gap between Training and Inference for Neural Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Neural Machine Translation generates target words sequentially while at inference it has to generate the entire sequence from scratch. |
| Approach: | They propose to use ground truth and inference to generate target words sequentially while at inference it has to generate the entire sequence from scratch. |
| Outcome: | Experiments on Chinese->English and WMT’14 English->German translation tasks show that the proposed model can achieve significant improvements on multiple datasets. |
Simple and Effective Noisy Channel Modeling for Neural Machine Translation (D19-1)
Copied to clipboard
| Challenge: | Previous work on noisy channel modeling relied on latent variable models that incrementally process the source and target sentence. |
| Approach: | They propose to use a standard sequence to sequence model which utilizes the entire source and target sentences to estimate posterior probability of a target sequence y given a source sequence x. |
| Outcome: | The proposed model outperforms direct models on German-English translations by up to 3.2 BLEU on four language pairs. |
Machine Translationese: Effects of Algorithmic Bias on Linguistic Complexity in Machine Translation (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing studies have shown that existing models amplify biases observed in training data. |
| Approach: | They propose to use MT and NLP to amplify biases observed in training data to investigate how bias amplification might affect language in a broader sense. |
| Outcome: | The proposed model amplifys biases observed in training data and could lead to an artificially impoverished language, the authors show. |
Overestimation of Syntactic Representation in Neural Language Models (2020.acl-main)
Copied to clipboard
| Challenge: | Several testing methodologies have been developed to probe models’ syntactic representations. |
| Approach: | They propose a method to determine syntactic structure by training a model on strings generated according to a template and testing its ability to distinguish between similar ones with different syntax. |
| Outcome: | The proposed method reproduces positive results with two non-syntactic baseline language models: an n-gram model and an LSTM model trained on scrambled inputs. |