| Challenge: | a novel scheme to perform word-level quality estimation is proposed for word-based quality estimation . authors propose a two-stage transfer learning procedure on augmented and human data . a Levenshtein Transformer can learn to post-edit without explicit supervision. |
| Approach: | They propose a novel scheme to use a Levenshtein Transformer to perform word-level quality estimation. |
| Outcome: | The proposed method performs better under data-constrained and unconstrained conditions than existing methods. |
Similar Papers
An Exploratory Analysis of Multilingual Word-Level Quality Estimation with Cross-Lingual Transformers (2021.acl-short)
Copied to clipboard
| Challenge: | Existing word-level quality estimation models require labelled data for each language pair and expensive maintenance. |
| Approach: | They propose to use multilingual QE models to generalise across languages . they propose to train models on other language pairs to predict word-level quality . |
| Outcome: | The proposed models generalise well across languages, making them more useful in real-world scenarios. |
TransQuest: Translation Quality Estimation with Cross-lingual Transformers (2020.coling-main)
Copied to clipboard
| Challenge: | Recent advances in the field of sentence-level quality estimation (QE) are based on neural-based architectures that require resourceintensive training. |
| Approach: | They propose a framework for sentence-level quality estimation based on cross-lingual transformers and use it to implement and evaluate two different neural architectures. |
| Outcome: | The proposed framework outperforms open-source QE frameworks when trained on WMT datasets and is very competitive in transfer learning settings. |
Investigating the Helpfulness of Word-Level Quality Estimation for Post-Editing Machine Translation Output (2021.emnlp-main)
Copied to clipboard
| Challenge: | Post-editing (PE) machine translation (MT) output can save time and reduce errors. |
| Approach: | They propose to use automatic word-level quality estimation to predict correctness of MT output to flag problematic output. |
| Outcome: | The proposed model is not good enough to support human translations, but is based on a visualization reflecting uncertainty of the model. |
Self-Supervised Quality Estimation for Machine Translation (2021.emnlp-main)
Copied to clipboard
Yuanhang Zheng, Zhixing Tan, Meng Zhang, Mieradilijiang Maimaiti, Huanbo Luan, Maosong Sun, Qun Liu, Yang Liu
| Challenge: | Training QE models require massive parallel data with hand-crafted quality annotations, which are time-consuming and labor-intensive to obtain. |
| Approach: | They propose a self-supervised method to evaluate machine-translated sentences without references by recovering masked target words. |
| Outcome: | The proposed method outperforms previous unsupervised methods on several QE tasks in different language pairs and domains. |
Sentence Level Human Translation Quality Estimation with Attention-based Neural Networks (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing methods for assessing translation quality rely on manual features and external knowledge. |
| Approach: | They propose to use a neural model without feature engineering to detect which parts in sentence pairs are most relevant for assessing quality. |
| Outcome: | The proposed model outperforms feature-based methods on a large human annotated dataset. |
deepQuest: A Framework for Neural-based Quality Estimation (C18-1)
Copied to clipboard
| Challenge: | Predicting Machine Translation (MT) quality has been limited to word and sentence-level prediction. |
| Approach: | They propose a framework that can generalize neural QE approaches to the level of documents. |
| Outcome: | The proposed framework outperforms state-of-the-art approaches on document-level quality estimates and is 40 times faster to train. |
Rethinking the Word-level Quality Estimation for Machine Translation from Human Judgement (2023.findings-acl)
Copied to clipboard
| Challenge: | Word-level Quality Estimation (QE) of Machine Translation aims to detect potential translation errors in the translated sentence without reference. |
| Approach: | They propose to use a human-generated translation judgment to generate a word-level quality estimate (QE) using a translation error rate toolkit to detect translation errors without reference. |
| Outcome: | The proposed dataset is more consistent with human judgment and confirms the effectiveness of the proposed tag-correcting strategies. |
Classification-based Quality Estimation: Small and Efficient Models for Real-world Applications (2021.emnlp-main)
Copied to clipboard
| Challenge: | Sentence-level Quality estimation (QE) is traditionally a regression task . but large multilingual contextualized language models are expensive and infeasible for real-world applications. |
| Approach: | They evaluate several model compression techniques for QE and find they are inefficient . they argue that a full model parameterization is required to achieve SoTA results . |
| Outcome: | The proposed models are poorly expressive in a regression task, the authors argue . they show that reframing QE as a classification problem and evaluating models would improve their performance in real-world applications. |
Are we Estimating or Guesstimating Translation Quality? (2020.acl-main)
Copied to clipboard
| Challenge: | A carefully engineered ensemble of pre-trained multilingual language models won the QE shared task at WMT19. |
| Approach: | They propose to use pre-trained multilingual language models to train quality estimation for machine translation. |
| Outcome: | A carefully engineered ensemble of pre-trained language models wins the QE shared task at WMT19. |
Transfer Fine-tuning for Quality Estimation of Text Simplification (2024.lrec-main)
Copied to clipboard
| Challenge: | Experimental results show that quality estimation of text simplification models can be improved on a small labeled corpus. |
| Approach: | They propose a method to train quality estimation of text simplification on a small-scale labeled corpus prior to fine-tuning pre-trained language models. |
| Outcome: | The proposed method improves quality estimation of text simplification on a small-scale labeled corpus. |