PreQuEL: Quality Estimation of Machine Translation Outputs in Advance (2022.emnlp-main)
Copied to clipboard
| Challenge: | A PreQuEL system predicts how well a given sentence will be translated without recourse to the actual translation. |
| Approach: | They propose a task that uses a model to predict how well a given sentence will be translated . they show that the model is sensitive to syntactic and semantic distinctions . |
| Outcome: | The proposed model improves on the Quality-Estimation task and on challenge sets and languages. |
Similar Papers
deepQuest: A Framework for Neural-based Quality Estimation (C18-1)
Copied to clipboard
| Challenge: | Predicting Machine Translation (MT) quality has been limited to word and sentence-level prediction. |
| Approach: | They propose a framework that can generalize neural QE approaches to the level of documents. |
| Outcome: | The proposed framework outperforms state-of-the-art approaches on document-level quality estimates and is 40 times faster to train. |
Are we Estimating or Guesstimating Translation Quality? (2020.acl-main)
Copied to clipboard
| Challenge: | A carefully engineered ensemble of pre-trained multilingual language models won the QE shared task at WMT19. |
| Approach: | They propose to use pre-trained multilingual language models to train quality estimation for machine translation. |
| Outcome: | A carefully engineered ensemble of pre-trained language models wins the QE shared task at WMT19. |
Sentence Level Human Translation Quality Estimation with Attention-based Neural Networks (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing methods for assessing translation quality rely on manual features and external knowledge. |
| Approach: | They propose to use a neural model without feature engineering to detect which parts in sentence pairs are most relevant for assessing quality. |
| Outcome: | The proposed model outperforms feature-based methods on a large human annotated dataset. |
Computer Assisted Translation with Neural Quality Estimation and Automatic Post-Editing (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Using neural machine translation to approximate human parity is difficult due to the lack of parallel training corpora. |
| Approach: | They propose an end-to-end deep learning framework for quality estimation and automatic post-editing of machine translation output. |
| Outcome: | The proposed framework achieves state-of-the-art performance on the English–German dataset and human translators can significantly expedite their post-editing processing with the model. |
Unsupervised Quality Estimation for Neural Machine Translation (2020.tacl-1)
Copied to clipboard
Marina Fomicheva, Shuo Sun, Lisa Yankovskaya, Frédéric Blain, Francisco Guzmán, Mark Fishel, Nikolaos Aletras, Vishrav Chaudhary, Lucia Specia
| Challenge: | Existing approaches require large amounts of expert annotated data, computation, and time for training. |
| Approach: | They propose an unsupervised approach to QE where no training is required . they use a dataset that enables work on both black-box and glass-box approaches . |
| Outcome: | The proposed approach rivals state-of-the-art supervised QE models in terms of correlation with human judgments of quality. |
Self-Supervised Quality Estimation for Machine Translation (2021.emnlp-main)
Copied to clipboard
Yuanhang Zheng, Zhixing Tan, Meng Zhang, Mieradilijiang Maimaiti, Huanbo Luan, Maosong Sun, Qun Liu, Yang Liu
| Challenge: | Training QE models require massive parallel data with hand-crafted quality annotations, which are time-consuming and labor-intensive to obtain. |
| Approach: | They propose a self-supervised method to evaluate machine-translated sentences without references by recovering masked target words. |
| Outcome: | The proposed method outperforms previous unsupervised methods on several QE tasks in different language pairs and domains. |
Understanding Pre-Editing for Black-Box Neural Machine Translation (2021.eacl-main)
Copied to clipboard
| Challenge: | a study has demonstrated the effectiveness of pre-editing for black-box neural MT, but a deep understanding of what it is and how it works for black box NMT is lacking. |
| Approach: | They investigated 6,652 instances of pre-editing across three translation directions, two MT systems and four text domains. |
| Outcome: | The proposed method can be used in MT systems with black-box neural MT (NMT) but it is not yet fully understood in the literature. |
Neural Machine Translation Quality and Post-Editing Performance (2021.emnlp-main)
Copied to clipboard
| Challenge: | a recent study has shown that MT post-editing can reduce translation quality and speed . a large-scale study involving 30 professional translators examined the relationship between MT performance and post-edited outputs. |
| Approach: | They examine the relationship between MT performance and post-editing time and quality . they use neural MT of high quality to improve translation quality based on phrase-based MT . |
| Outcome: | The proposed model is not stable predictor of time or quality, the authors say . they find that better MT systems lead to fewer changes in the sentences . |
Transfer Fine-tuning for Quality Estimation of Text Simplification (2024.lrec-main)
Copied to clipboard
| Challenge: | Experimental results show that quality estimation of text simplification models can be improved on a small labeled corpus. |
| Approach: | They propose a method to train quality estimation of text simplification on a small-scale labeled corpus prior to fine-tuning pre-trained language models. |
| Outcome: | The proposed method improves quality estimation of text simplification on a small-scale labeled corpus. |
QUAK: A Synthetic Quality Estimation Dataset for Korean-English Neural Machine Translation (2022.coling-1)
Copied to clipboard
| Challenge: | despite its high utility, there are limitations concerning manual QE data creation. |
| Approach: | They propose to generate a Korean-English QE dataset that is fully automatic . they find that the algorithm is more accurate and faster than manual QE . |
| Outcome: | The proposed datasets show that they scale up to 1.58M and 6.58M, respectively, and show that the results are significantly better when compared to the previous datasets. |