Backtranslation Feedback Improves User Confidence in MT, Not Quality (2021.naacl-main)
Copied to clipboard
Vilém Zouhar, Michal Novák, Matúš Žilinec, Ondřej Bojar, Mateo Obregón, Robin L. Hill, Frédéric Blain, Marina Fomicheva, Lucia Specia, Lisa Yankovskaya
| Challenge: | Inbound translation is a modern need for which the user experience has significant room for improvement, beyond the basic machine translation facility. |
| Approach: | They propose to provide cues that indicate the quality of MT output as well as suggest possible rephrasing of the source language. |
| Outcome: | The proposed feedback module increases user confidence in the produced translation, but not the objective quality. |
Similar Papers
Outbound Translation User Interface Ptakopět: A Pilot Study (2020.lrec-1)
Copied to clipboard
| Challenge: | a task called outbound translation is not uncommon for Internet users to have to produce a text in a foreign language they have very little knowledge of and are unable to verify the translation quality. |
| Approach: | They propose an open-source modular system to inspect human interaction with machine translation systems enhanced with additional subsystems such as backward translation and quality estimation. |
| Outcome: | The proposed system is able to produce a text in a foreign language with minimal knowledge and is compared with MT systems of mid-range quality. |
Tagged Back-translation Revisited: Why Does It Really Work? (2020.acl-main)
Copied to clipboard
| Challenge: | In this paper, we show that neural machine translation systems trained on large back-translated data overfit some of the characteristics of machine-transcribed texts. |
| Approach: | They propose to add a tag to back-translations to help distinguish back-translated data from original parallel training data. |
| Outcome: | The proposed tag helps the system distinguish back-translated data from original parallel training data and is as effective as a tag in high-resource training. |
Physician Detection of Clinical Harm in Machine Translation: Quality Estimation Aids in Reliance and Backtranslation Identifies Critical Errors (2023.emnlp-main)
Copied to clipboard
| Challenge: | a major challenge in the practical use of Machine Translation (MT) is that users lack information on translation quality to make informed decisions about how to rely on outputs. |
| Approach: | They evaluate quality estimation feedback in vivo with a human study in a medical setting. |
| Outcome: | The proposed method improves appropriate reliance on MT, but backtranslation helps detect harmful errors. |
Should I Share this Translation? Evaluating Quality Feedback for User Reliance on Machine Translation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on the impact of feedback on human decision-making are limited as people are not equipped to assess the quality of AI predictions. |
| Approach: | They compare the quality of MT inputs and outputs with explicit and implicit feedbacks that directly give users an assessment of translation quality using error highlights and LLM explanations. |
| Outcome: | The proposed model improves decision accuracy and appropriate reliance by using error highlights and explanations, and by using backtranslation and question–answer tables. |
Toward Machine Translation Literacy: How Lay Users Perceive and Rely on Imperfect Translations (2025.emnlp-main)
Copied to clipboard
Yimin Xiao, Yongle Zhang, Dayeon Ki, Calvin Bao, Marianna J. Martindale, Charlotte Vaughn, Ge Gao, Marine Carpuat
| Challenge: | Using machine translation tools for everyday tasks is becoming more commonplace, but a lack of evaluation strategies and alternatives can cause users to over-rely on it. |
| Approach: | They propose to use MT evaluation techniques to promote MT quality and MT literacy among its users. |
| Outcome: | The findings highlight the need for evaluation and NLP explanation techniques to promote MT quality and MT literacy among its users. |
On The Evaluation of Machine Translation Systems Trained With Back-Translation (2020.acl-main)
Copied to clipboard
| Challenge: | Back-translation is a data augmentation technique that can be used to improve neural machine translation systems. |
| Approach: | They propose to combine back-translation with a language model score to measure fluency. |
| Outcome: | The proposed method improves translation quality of natural text and translationese according to professional translators. |
Improving Back-Translation with Uncertainty-based Confidence Estimation (D19-1)
Copied to clipboard
| Challenge: | Despite the success of low-resource neural machine translation, there is a data scarcity problem in many languages . large-scale, high-quality, and widecoverage bilingual corpora do not exist for most language pairs . |
| Approach: | They propose to quantify confidence of NMT models based on model uncertainty . they propose to use uncertainty-based confidence measures to improve back-translation . |
| Outcome: | The proposed model outperforms conventional statistical machine translation (SMT) on Chinese-English and English-German translation tasks. |
Measuring Uncertainty in Translation Quality Evaluation (TQE) (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing automated tools are not good enough to evaluate translation quality . existing tools are often accused of having low reliability and agreement . |
| Approach: | They propose to use a method to accurately estimate the confidence intervals depending on the sample size of the translated text. |
| Outcome: | The proposed method aims to estimate the confidence intervals (CITATION) depending on the sample size of the translated text, e.g. the amount of words or sentences, that needs to be processed on TQE workflow step for confident and reliable evaluation of overall translation quality. |
Rethinking Round-Trip Translation for Machine Translation Evaluation (2023.findings-acl)
Copied to clipboard
| Challenge: | Automatic evaluation methods for translation often require model training and the availability of parallel corpora limits their applicability to low-resource settings. |
| Approach: | They revisit the statistical machine translation technique and use it to improve translation quality. |
| Outcome: | The proposed method improves translation quality estimation models and identifies adversarial competitors in shared tasks via cross-system verification. |
Online Learning Meets Machine Translation Evaluation: Finding the Best Systems with the Least Human Effort (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods to evaluate multiple systems are expensive and require human evaluators. |
| Approach: | They propose a novel online learning approach that dynamically converges to the top-3 ranked systems for the language pairs considered by taking advantage of human feedback. |
| Outcome: | The proposed approach converges to the top-3 ranked systems for the language pairs considered despite the lack of human feedback for many translations. |