Long-Short Term Masking Transformer: A Simple but Effective Baseline for Document-level Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in deep learning have led to significant improvement of document-level neural machine translation (NMT). |
| Approach: | They propose a long-short term masking self-attention on top of the standard transformer to capture the long-range dependence and reduce the propagation of errors. |
| Outcome: | The proposed model captures the long-range dependence and reduces errors on two publicly available document-level datasets. |
Similar Papers
Rethinking Document-level Neural Machine Translation (2022.findings-acl)
Copied to clipboard
| Challenge: | Neural machine translation models are weak enough for document-level translation . current models only translate sentences individually, resulting in poor document coherence . |
| Approach: | They propose to use the original Transformer model to test document-level neural machine translation . they find that the original transformer models can achieve strong results for document translation if trained properly . |
| Outcome: | The proposed model outperforms sentence-level models on nine datasets and two sentence- level datasets across six languages. |
G-Transformer for Document-Level Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing work extends translation unit from single sentence to multiple sentences. |
| Approach: | They propose to introduce locality assumption as an inductive bias into Transformer and reduce the hypothesis space of attention from target to source. |
| Outcome: | The proposed model achieves state-of-the-art BLEU scores on three benchmark datasets. |
Learn To Remember: Transformer with Recurrent Memory for Document-Level Machine Translation (2022.findings-naacl)
Copied to clipboard
| Challenge: | Recent studies have shown that the effective use of contextual information between sentences can achieve better performance in document-level machine translation. |
| Approach: | They propose a recurrent memory unit to the Transformer to support the information exchange between the sentence and previous context. |
| Outcome: | The proposed model outperforms the previous work on TED and News by 0.91 s-BLEU and 1.49 d-BLUE on average. |
When and Why is Document-level Context Useful in Neural Machine Translation? (D19-65)
Copied to clipboard
| Challenge: | Recent advances in document-level NMT focus on sophisticated integration of the context, explaining its improvement with only a few selected examples or targeted test sets. |
| Approach: | They extensively quantify the causes of improvements by a document-level model in general test sets, clarifying the limit of the usefulness of document- level context in NMT. |
| Outcome: | The proposed model is not interpretable as utilizing the context, and a long context is not helpful for NMT. |
Improving the Transformer Translation Model with Document-Level Context (D18-1)
Copied to clipboard
| Challenge: | Existing models for document-level context translation ignore documentlevel context. |
| Approach: | They propose a document-level context encoder to represent document- level context and integrate it into the Transformer model. |
| Outcome: | Experiments on NIST Chinese-English and IWSLT French-English datasets show that the proposed translation model outperforms the Transformer model significantly. |
Multi-Hop Transformer for Document-Level Machine Translation (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to document-level neural machine translation (NMT) simply introduce the representations of context sentences without explicitly characterizing the inter-sentence reasoning process. |
| Approach: | They propose a novel multi-hop Transformer which explicitly models the human-like draft-editing and reasoning process by attending to multiple antecedent sentences iteratively. |
| Outcome: | Experiments on four widely used document translation tasks show that the proposed model significantly improves document-level translation performance and tackles discourse phenomena such as coreference error and the problem of polysemy. |
A Simple and Effective Approach to Coverage-Aware Neural Machine Translation (P18-2)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) models are used to solve translation problems using long-term models. |
| Approach: | They propose a method to seek a better balance between model confidence and length preference for Neural Machine Translation. |
| Outcome: | The proposed model improves on Chinese-English and English-German translation tasks. |
Corpora for Document-Level Neural Machine Translation (2020.lrec-1)
Copied to clipboard
| Challenge: | Document-level machine translation models translate sentences in isolation, but there are three main problems for document-level models. |
| Approach: | They propose to use document-level machine translation to capture discourse dependencies across sentences by considering a document as a whole. |
| Outcome: | The proposed method captures discourse dependencies across sentences by considering a document as a whole. |
On Search Strategies for Document-Level Neural Machine Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | Document-level neural machine translation models produce a more consistent output across a document . however, the exact decoding strategy is often not described and not mentioned at all. |
| Approach: | They propose to use standard automatic metrics and specific linguistic phenomena to compare different decoding schemes. |
| Outcome: | The proposed decoding strategies perform similar to each other on three standard document-level translation benchmarks. |
Document-Level Neural Machine Translation with Hierarchical Attention Networks (D18-1)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) can be improved by including document-level contextual information. |
| Approach: | They propose a hierarchical attention model that captures document-level contextual information and conditioning on the NMT model’s own hidden states. |
| Outcome: | The proposed model improves the BLEU score over a strong NMT baseline with the state-of-the-art in context-aware methods and that both the encoder and decoder benefit from context in complementary ways. |