Papers with ESA
TRANSLATIONCORRECT: A Unified Framework for Machine Translation Post-Editing with Predictive Error Assistance (2025.acl-demo)
Copied to clipboard
| Challenge: | Current workflows for machine translation (MT) post-editing and research data collection are inefficient and time-consuming. |
| Approach: | They propose a framework that combines MT and error prediction within a single environment. |
| Outcome: | **TranslationCorrect** exports high-quality span-based annotations in the Error Span Annotation format, using an error taxonomy inspired by Multidimensional Quality Metrics (MQM). |
AI-Assisted Human Evaluation of Machine Translation (2025.naacl-long)
Copied to clipboard
| Challenge: | Annotation metrics are misaligned with the ideal measure of text quality and human evaluation remains the most accurate, reliable, and ultimate standard. |
| Approach: | They propose an annotation protocol that helps annotators mark erroneous parts of the translation and assign a final score. |
| Outcome: | The proposed protocol reduces the time per span annotation by half . the method reduces annotation budget by 25% with filtering of examples that the AI deems to be likely to be correct. |
Refined Assessment for Translation Evaluation: Rethinking Machine Translation Evaluation in the Era of Human-Level Systems (2025.findings-emnlp)
Copied to clipboard
Dmitry Popov, Vladislav Negodin, Ekaterina Enikeeva, Iana Matrosova, Nikolay Karpachev, Max Ryabinin
| Challenge: | Currently, traditional evaluation methods struggle to detect subtle translation errors. |
| Approach: | They propose to use a dataset of human evaluations for English–Russian translations created by professional linguists to enable consistent and rich annotation. |
| Outcome: | The proposed protocol allows expert assessments without time pressure to yield substantially different results from standard evaluations. |