Translationese-index: Using Likelihood Ratios for Graded and Generalizable Measurement of Translationese (2025.emnlp-main)
Copied to clipboard
Yikang Liu, Wanyang Zhang, Yiming Wang, Jialong Tang, Pei Zhang, Baosong Yang, Fei Huang, Rui Wang, Hai Hu
| Challenge: | Translationese is a linguistic property that is often introduced in the translation process that is different from those of original texts. |
| Approach: | They propose to use synthesized translations and translations in the wild to evaluate T-index's generalizability in cross-domain settings and its validity against human judgments. |
| Outcome: | The proposed measure can generalize to unseen genres, authors, and language pairs. |
Similar Papers
Lost in Translation, and Found: Detecting and Interpreting Translation Effects (2026.acl-long)
Copied to clipboard
Shira Wein, Anna Serbina, Jiyuan Ji, Nathan Wolf, Jason DeGraaff, Prajakta Kini, Maria Leonor Pacheco
| Challenge: | Translationese refers to the statistical patterns that distinguish translated texts from original texts. |
| Approach: | They analyze linguistic features which enable our model to achieve high accuracy by a collection of linguistic characteristics and pretrained neural models pick up these features without any fine-tuning. |
| Outcome: | The proposed model achieves high accuracy with a set of linguistic features that correspond to translationese theories and pretrained neural models pick up these features without any fine-tuning. |
Lexicogrammatic translationese across two targets and competence levels (2020.lrec-1)
Copied to clipboard
| Challenge: | a specificity of translations with English as a source language produced by students and professional translators is investigated by genre-comparable data from a number of parallel and comparable corpora. |
| Approach: | They propose to use genre-comparable data to explore the specificity of translations . they use a set of human-interpretable lexicogrammatic translationese indicators . |
| Outcome: | The proposed feature set can reliably distinguish translations and non-translations regardless of the language pair and translation variety. |
Are we Estimating or Guesstimating Translation Quality? (2020.acl-main)
Copied to clipboard
| Challenge: | A carefully engineered ensemble of pre-trained multilingual language models won the QE shared task at WMT19. |
| Approach: | They propose to use pre-trained multilingual language models to train quality estimation for machine translation. |
| Outcome: | A carefully engineered ensemble of pre-trained language models wins the QE shared task at WMT19. |
Can Automatic Metrics Assess High-Quality Translations? (2024.emnlp-main)
Copied to clipboard
| Challenge: | a recent human evaluation study found that translations produced by current MT systems achieve very high-quality scores when judged by humans on a direct assessment scale of 0 to 100. |
| Approach: | They stress-test the ability of current translation quality metrics to detect correct translations . they show that current metrics often over or underestimate translation quality . |
| Outcome: | The proposed method overestimates translation quality, the authors show . they show that current metrics often overestimate translation quality . |
An Exploratory Analysis of Multilingual Word-Level Quality Estimation with Cross-Lingual Transformers (2021.acl-short)
Copied to clipboard
| Challenge: | Existing word-level quality estimation models require labelled data for each language pair and expensive maintenance. |
| Approach: | They propose to use multilingual QE models to generalise across languages . they propose to train models on other language pairs to predict word-level quality . |
| Outcome: | The proposed models generalise well across languages, making them more useful in real-world scenarios. |
Statistical Power and Translationese in Machine Translation Evaluation (2020.emnlp-main)
Copied to clipboard
| Challenge: | a recent paper argues that translationese has been used to describe features of translated text . a translationed text can be more explicit than the original source, authors say . authors recommend reverse-created test data be omitted from future evaluations . |
| Approach: | They propose to omit translationese from future machine translation evaluations . they also re-evaluate a past evaluation claiming human-parity of MT . |
| Outcome: | The proposed analysis shows that translationese does not affect machine translation evaluations. |
Rethinking the Word-level Quality Estimation for Machine Translation from Human Judgement (2023.findings-acl)
Copied to clipboard
| Challenge: | Word-level Quality Estimation (QE) of Machine Translation aims to detect potential translation errors in the translated sentence without reference. |
| Approach: | They propose to use a human-generated translation judgment to generate a word-level quality estimate (QE) using a translation error rate toolkit to detect translation errors without reference. |
| Outcome: | The proposed dataset is more consistent with human judgment and confirms the effectiveness of the proposed tag-correcting strategies. |
Translationese as a Language in “Multilingual” NMT (2020.acl-main)
Copied to clipboard
| Challenge: | Recent work examines the impact of translationese in machine translation evaluation using the WMT evaluation campaign. |
| Approach: | They propose to use a sentence-level classifier to distinguish translationese from original target text to generate a machine translation model that can produce more natural outputs at test time. |
| Outcome: | The proposed model produces more natural outputs at test time, yielding gains in human evaluation scores on accuracy and fluency. |
Fine-Tuned Machine Translation Metrics Struggle in Unseen Domains (2024.acl-short)
Copied to clipboard
| Challenge: | a new dataset examines whether fine-tuned metrics are robust to domain shifts between training and inference. |
| Approach: | They use an annotated multidimensional quality metrics dataset to examine whether they are robust to domain shifts between training and inference. |
| Outcome: | The proposed metrics exhibit a substantial performance drop in the unseen domain scenario compared to metrics that rely on the surface form and pre-trained metrics that are not fine-tuned on MT quality judgments. |
Understanding Cross-Lingual Alignment—A Survey (2024.findings-acl)
Copied to clipboard
| Challenge: | Cross-lingual alignment is the meaningful similarity of representations across languages in multilingual language models. |
| Approach: | They propose a taxonomy of methods to improve cross-lingual alignment . they argue that an effective trade-off between language-neutral and language-specific information is key . |
| Outcome: | The proposed methods can be applied to encoder models and encoder-decoder-only models . they show that language-neutral and language-specific information is key . |