A Simple and Effective Unified Encoder for Document-Level Machine Translation (2020.acl-main)
Copied to clipboard
| Challenge: | Existing models for document-level machine translation use two separate encoders to model the source sentences and document- level contexts. |
| Approach: | They propose a unified encoder that can outperform existing models of dual-encoder models . they propose to use document-level contexts to model the interaction between the contexts and the source sentences . |
| Outcome: | The proposed model outperforms baseline models of dual-encoder models in terms of BLEU and METEOR scores. |
Similar Papers
Document-Level Neural Machine Translation Using BERT as Context Encoder (2020.aacl-srw)
Copied to clipboard
| Challenge: | Large-scale pre-trained representations such as BERT have been widely used in many natural language understanding tasks. |
| Approach: | They propose to use BERT as a context encoder to achieve document-level contextual information, which is then integrated into both the encoder and decoder. |
| Outcome: | The proposed model outperforms strong document-level machine translation baselines on BLEU score and captures document- level context information to boost translation performance. |
Improving the Transformer Translation Model with Document-Level Context (D18-1)
Copied to clipboard
| Challenge: | Existing models for document-level context translation ignore documentlevel context. |
| Approach: | They propose a document-level context encoder to represent document- level context and integrate it into the Transformer model. |
| Outcome: | Experiments on NIST Chinese-English and IWSLT French-English datasets show that the proposed translation model outperforms the Transformer model significantly. |
Rethinking Document-level Neural Machine Translation (2022.findings-acl)
Copied to clipboard
| Challenge: | Neural machine translation models are weak enough for document-level translation . current models only translate sentences individually, resulting in poor document coherence . |
| Approach: | They propose to use the original Transformer model to test document-level neural machine translation . they find that the original transformer models can achieve strong results for document translation if trained properly . |
| Outcome: | The proposed model outperforms sentence-level models on nine datasets and two sentence- level datasets across six languages. |
Does Multi-Encoder Help? A Case Study on Context-Aware Neural Machine Translation (2020.acl-main)
Copied to clipboard
| Challenge: | In encoder-decoder neural models, multiple encoders are used to represent contextual information in addition to the individual sentence. |
| Approach: | They propose to use multiple context encoders to encode the individual sentences in document-level neural machine translation (NMT) They propose a noisy dropout setup and a single-encoder approach to encode context sentences. |
| Outcome: | The proposed approach encodes the context and the current sentence without contexts. |
Multi-Source Text Classification for Multilingual Sentence Encoder with Machine Translation (2024.naacl-srw)
Copied to clipboard
| Challenge: | Pre-trained multilingual sentence encoders suffer from performance degradation for non-English languages. |
| Approach: | They propose a method of machine translating a source sentence into English and then inputting it together with the source sentence in a multi-source manner. |
| Outcome: | The proposed method improves the performance of pre-trained multilingual sentence encoders in Japanese on sentiment analysis and topic classification tasks. |
Revisiting Context Choices for Context-aware Machine Translation (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent work has cast doubt on whether context-aware machine translation models learn useful signals from context or are improvements in automatic evaluation metrics just a side-effect. |
| Approach: | They propose to use separate encoders for source sentence and context as multiple sources for one target sentence to train context-aware machine translation models. |
| Outcome: | The proposed model improves translation quality even with empty lines as context, but the correct context improves it and random out-of-domain context degrades it. |
On the differences between BERT and MT encoder spaces and how to address them in translation tasks (2021.acl-srw)
Copied to clipboard
| Challenge: | Various studies show that pretrained language models cannot replace encoders in neural machine translation despite their success in other tasks. |
| Approach: | They propose a supervised transformation from one into the other to improve the applicability of BERT in neural machine translation. |
| Outcome: | The proposed transformations show that they cannot replace encoders in MT despite their success in other tasks. |
Diverse Pretrained Context Encodings Improve Document Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing models for sentence-level sequence-to-sequence translations do not use extra-sentential information. |
| Approach: | They propose a sentence-level sequence-to-sequence transformer with multiple pre-trained context signals. |
| Outcome: | The proposed model outperforms existing models on Chinese-English and English-German tasks. |
G-Transformer for Document-Level Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing work extends translation unit from single sentence to multiple sentences. |
| Approach: | They propose to introduce locality assumption as an inductive bias into Transformer and reduce the hypothesis space of attention from target to source. |
| Outcome: | The proposed model achieves state-of-the-art BLEU scores on three benchmark datasets. |
Hierarchical Modeling of Global Context for Document-Level Neural Machine Translation (D19-1)
Copied to clipboard
| Challenge: | Document-level machine translation (MT) remains challenging due to the difficulty in efficiently using document context. |
| Approach: | They propose a hierarchical model to learn document context for document-level neural machine translation . they use a sentence encoder to capture intra-sentence dependencies and a document encoder . |
| Outcome: | The proposed model significantly improves document-level translation performance over strong baselines. |