Exploring Document-Level Literary Machine Translation with Parallel Paragraphs from World Literature (2022.emnlp-main)
Copied to clipboard
Katherine Thai, Marzena Karpinska, Kalpesh Krishna, Bill Ray, Moira Inghilleri, John Wieting, Mohit Iyyer
| Challenge: | Literary translation is a culturally significant task, but it is bottlenecked by the small number of qualified literary translators . a dataset of non-English language novels is used to study literary MT . |
| Approach: | They use a dataset of non-English language novels aligned to human and automatic English translations to study literary MT. |
| Outcome: | The proposed model prefers human translations over machine translations at a rate of 84% . state-of-the-art MT metrics do not correlate with preferences, the study finds . |
Similar Papers
How Good Are LLMs for Literary Translation, Really? Literary Translation Evaluation with Humans and LLMs (2025.naacl-long)
Copied to clipboard
| Challenge: | Recent research has focused on literary machine translation (MT) but evaluation of literary MT remains an open problem. |
| Approach: | They propose a paragraph-level parallel corpus containing verified human translations and 13k evaluated sentences across four language pairs. |
| Outcome: | The proposed corpus compares human evaluations with students and professionals . it shows that the adequacy of human evaluation is controlled by two factors . |
Discourse-Centric Evaluation of Document-level Machine Translation with a New Densely Annotated Parallel Corpus of Novels (2023.acl-long)
Copied to clipboard
| Challenge: | Several recent papers claim to have achieved human parity at sentence-level machine translation. |
| Approach: | They propose to use a dataset with rich discourse annotations to evaluate MT performance . they find that MT outputs differ fundamentally from human translations in terms of latent discourse structures. |
| Outcome: | The proposed dataset builds upon the large-scale parallel corpus BWB . it covers 15,095 entity mentions in both languages and compares them to human translations . |
Has Machine Translation Achieved Human Parity? A Case for Document-level Evaluation (D18-1)
Copied to clipboard
| Challenge: | Recent research suggests that neural machine translation achieves parity with professional human translation on the WMT Chinese–English news translation task. |
| Approach: | They empirically test neural machine translation on a Chinese–English news translation task . they show human raters prefer human over machine translation when evaluating documents . |
| Outcome: | The proposed method shows that human translators prefer document-level evaluation over machine translation . the results highlight the need to shift towards document- level evaluation as machine translation improves . |
Toward Machine Translation Literacy: How Lay Users Perceive and Rely on Imperfect Translations (2025.emnlp-main)
Copied to clipboard
Yimin Xiao, Yongle Zhang, Dayeon Ki, Calvin Bao, Marianna J. Martindale, Charlotte Vaughn, Ge Gao, Marine Carpuat
| Challenge: | Using machine translation tools for everyday tasks is becoming more commonplace, but a lack of evaluation strategies and alternatives can cause users to over-rely on it. |
| Approach: | They propose to use MT evaluation techniques to promote MT quality and MT literacy among its users. |
| Outcome: | The findings highlight the need for evaluation and NLP explanation techniques to promote MT quality and MT literacy among its users. |
Revisiting Machine Translation for Cross-lingual Classification (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent work in cross-lingual learning has pivoted around multilingual models, which are typically pretrained on unlabeled corpora in multiple languages using some form of language modeling objective. |
| Approach: | They propose to use a stronger machine translation system to mitigat mismatch between training on original text and running inference on machine translated text. |
| Outcome: | The proposed approach is highly task dependent and calls into question the dominance of multilingual models for cross-lingual classification. |
An Interdisciplinary Approach to Human-Centered Machine Translation (2025.emnlp-main)
Copied to clipboard
Marine Carpuat, Omri Asscher, Kalika Bali, Luisa Bentivogli, Fred Blain, Lynne Bowker, Monojit Choudhury, Hal Daumé Iii, Kevin Duh, Ge Gao, Alvin C Grissom II, Marzena Karpinska, Elaine C Khoong, William D. Lewis, Andre Martins, Mary Nurminen, Douglas W. Oard, Maja Popovic, Michel Simard, François Yvon
| Challenge: | Despite progress in MT, a gap persists between how the technology is developed and how it is used in real-world contexts. |
| Approach: | They propose a human-centered approach to machine translation (MT) they argue that MT should be evaluated with diverse goals and contexts of use . |
| Outcome: | The proposed approach emphasizes alignment of evaluation and design with diverse communicative goals and contexts of use. |
Machine Translation of Restaurant Reviews: New Corpus for Domain Adaptation and Robustness (D19-56)
Copied to clipboard
Alexandre Berard, Ioan Calapodescu, Marc Dymetman, Claude Roux, Jean-Luc Meunier, Vassilina Nikoulina
| Challenge: | BLEU: MT is a very robust and efficient way to translate user-generated content. |
| Approach: | They propose a task to encourage research on MT robustness and domain adaptation . they ask professionals to translate 11.5k french 4SQ reviews to English . |
| Outcome: | The proposed task improves on the existing MT systems in a real-world scenario . the proposed methods improve translation accuracy and sentiment analysis . |
LiTransProQA: An LLM-based Literary Translation Evaluation Metric with Professional Question Answering (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing evaluation metrics for literature prioritize mechanical accuracy over artistic expression . this bias could result in an irreversible decline in translation quality and cultural authenticity . |
| Approach: | They propose a novel, reference-free, LLM-based question-answering framework for literary translation evaluation. |
| Outcome: | a novel, reference-free, LLM-based question-answering framework is developed for literary translation evaluation. |
Challenges in Context-Aware Neural Machine Translation (2023.emnlp-main)
Copied to clipboard
| Challenge: | despite well-reasoned intuitions, most context-aware neural machine translation models show only modest improvements over sentence-level systems. |
| Approach: | They propose a more realistic setting for document-level translation called paragraph-to-paragraph (PARA2PARA) they collect a dataset of Chinese-English novels to promote future research . |
| Outcome: | The proposed model improves translation quality across document-level metrics and discourse phenomena. |
Upping the Ante: Towards a Better Benchmark for Chinese-to-English Machine Translation (L18-1)
Copied to clipboard
| Challenge: | Currently, there is no widely accepted standard for evaluation of machine translation (MT) for Chinese-to-English translation, there are no standard for standardized training sets, development sets, and test sets. |
| Approach: | They propose to use Chinese-to-English machine translation as a benchmark . they build a highly competitive state-of-the-art MT system that outperforms reported results . |
| Outcome: | The proposed system outperforms reported results on NIST OpenMT test sets in almost all papers published in major conferences and journals in computational linguistics and artificial intelligence in the past 11 years. |