Intermediate Self-supervised Learning for Machine Translation Quality Estimation (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods for machine translation quality estimation (QE) rely on annotated data. |
| Approach: | They propose a self-supervised learning task for machine translation (MT) that orients a pre-trained model towards the target task. |
| Outcome: | The proposed method outperforms existing methods on English-to-German and English- to-Russian translation directions and is comparable to existing models. |
Similar Papers
Self-Supervised Quality Estimation for Machine Translation (2021.emnlp-main)
Copied to clipboard
Yuanhang Zheng, Zhixing Tan, Meng Zhang, Mieradilijiang Maimaiti, Huanbo Luan, Maosong Sun, Qun Liu, Yang Liu
| Challenge: | Training QE models require massive parallel data with hand-crafted quality annotations, which are time-consuming and labor-intensive to obtain. |
| Approach: | They propose a self-supervised method to evaluate machine-translated sentences without references by recovering masked target words. |
| Outcome: | The proposed method outperforms previous unsupervised methods on several QE tasks in different language pairs and domains. |
Are we Estimating or Guesstimating Translation Quality? (2020.acl-main)
Copied to clipboard
| Challenge: | A carefully engineered ensemble of pre-trained multilingual language models won the QE shared task at WMT19. |
| Approach: | They propose to use pre-trained multilingual language models to train quality estimation for machine translation. |
| Outcome: | A carefully engineered ensemble of pre-trained language models wins the QE shared task at WMT19. |
deepQuest: A Framework for Neural-based Quality Estimation (C18-1)
Copied to clipboard
| Challenge: | Predicting Machine Translation (MT) quality has been limited to word and sentence-level prediction. |
| Approach: | They propose a framework that can generalize neural QE approaches to the level of documents. |
| Outcome: | The proposed framework outperforms state-of-the-art approaches on document-level quality estimates and is 40 times faster to train. |
Classification-based Quality Estimation: Small and Efficient Models for Real-world Applications (2021.emnlp-main)
Copied to clipboard
| Challenge: | Sentence-level Quality estimation (QE) is traditionally a regression task . but large multilingual contextualized language models are expensive and infeasible for real-world applications. |
| Approach: | They evaluate several model compression techniques for QE and find they are inefficient . they argue that a full model parameterization is required to achieve SoTA results . |
| Outcome: | The proposed models are poorly expressive in a regression task, the authors argue . they show that reframing QE as a classification problem and evaluating models would improve their performance in real-world applications. |
PreQuEL: Quality Estimation of Machine Translation Outputs in Advance (2022.emnlp-main)
Copied to clipboard
| Challenge: | A PreQuEL system predicts how well a given sentence will be translated without recourse to the actual translation. |
| Approach: | They propose a task that uses a model to predict how well a given sentence will be translated . they show that the model is sensitive to syntactic and semantic distinctions . |
| Outcome: | The proposed model improves on the Quality-Estimation task and on challenge sets and languages. |
Sentence Level Human Translation Quality Estimation with Attention-based Neural Networks (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing methods for assessing translation quality rely on manual features and external knowledge. |
| Approach: | They propose to use a neural model without feature engineering to detect which parts in sentence pairs are most relevant for assessing quality. |
| Outcome: | The proposed model outperforms feature-based methods on a large human annotated dataset. |
Translation Error Detection as Rationale Extraction (2022.findings-acl)
Copied to clipboard
| Challenge: | Recent Quality Estimation models rely on translation errors to predict overall sentence quality, but detecting specific errors is a more challenging task. |
| Approach: | They propose to use a semi-supervised method to detect translation errors by attribution of relevance scores to inputs to explain model predictions. |
| Outcome: | The proposed method can detect translation errors and is compared with human models using a set of feature attribution methods. |
Rethinking the Word-level Quality Estimation for Machine Translation from Human Judgement (2023.findings-acl)
Copied to clipboard
| Challenge: | Word-level Quality Estimation (QE) of Machine Translation aims to detect potential translation errors in the translated sentence without reference. |
| Approach: | They propose to use a human-generated translation judgment to generate a word-level quality estimate (QE) using a translation error rate toolkit to detect translation errors without reference. |
| Outcome: | The proposed dataset is more consistent with human judgment and confirms the effectiveness of the proposed tag-correcting strategies. |
Quality Estimation-Assisted Automatic Post-Editing (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing APE and QE combination strategies have not shown significant performance gains in the field of automatic post-editing (APE). |
| Approach: | They propose to train a model on APE and QE tasks to improve the APE performance by using a multi-task learning methodology that treats both tasks as a 'bargaining game' they also investigate various existing combination strategies and show that their approach achieves state-of-the-art performance for a ‘distant’ language pair, viz., English-Marathi. |
| Outcome: | The proposed model improves on two different language pairs, viz., English-Marathi and English-German. |
Levenshtein Training for Word-level Quality Estimation (2021.emnlp-main)
Copied to clipboard
| Challenge: | a novel scheme to perform word-level quality estimation is proposed for word-based quality estimation . authors propose a two-stage transfer learning procedure on augmented and human data . a Levenshtein Transformer can learn to post-edit without explicit supervision. |
| Approach: | They propose a novel scheme to use a Levenshtein Transformer to perform word-level quality estimation. |
| Outcome: | The proposed method performs better under data-constrained and unconstrained conditions than existing methods. |