Towards Precise Localization of Critical Errors in Machine Translation (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing critical error detection focuses on identifying sentence-level errors, leaving the precise localization of such errors unaddressed. |
| Approach: | They propose a task to detect critical errors at a fine-grained level in machine translation sentences. |
| Outcome: | The proposed method outperforms existing methods and LLMs in English to Korean translations. |
Similar Papers
Detecting Critical Errors Considering Cross-Cultural Factors in English-Korean Translation (2024.lrec-main)
Copied to clipboard
Sugyeong Eo, Jungwoo Lim, Chanjun Park, DaHyun Jung, Seonmin Koo, Hyeonseok Moon, Jaehyung Seo, Heuiseok Lim
| Challenge: | Recent machine translation systems overcome language barriers for a wide range of users, yet they carry the risk of catastrophic meaning deviations. |
| Approach: | They introduce a culture-aware "Politeness" type for detecting critical translation errors . they also provide multiclass labels for critical error detection and critical error type classification . |
| Outcome: | Empirical results show that the proposed method outperforms baselines in both tasks. |
Explainable CED: A Dataset for Explainable Critical Error Detection in Machine Translation (2024.naacl-srw)
Copied to clipboard
| Challenge: | Existing studies of critical error detection lack content addressing the causes of catastrophic errors. |
| Approach: | They propose a dataset that introduces the attributes of error explanation and correction regarding critical errors. |
| Outcome: | The proposed dataset reduces time costs and mitigates human annotation bias. |
An Evaluation Resource for Grounding Translation Errors (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Current fine-grained error analyses do not ground the errors to the reasons why the annotated text spans are erroneous. |
| Approach: | They use a bi-directional grounding scheme to ground erroneous text in two directions . if the error spans of both directions are consistent, the explanation is valid . |
| Outcome: | The proposed grounding process improves translation error detection significantly. |
Revisit Automatic Error Detection for Wrong and Missing Translation – A Supervised Approach (D19-1)
Copied to clipboard
| Challenge: | Current machine translation techniques are bottlenecked by adequacy issues . we propose automatic detection of missing and wrong translations . |
| Approach: | They propose automatic detection of adequacy errors in MT hypothesis for MT model evaluation by annotating missing and wrong translations in 15000 Chinese-English translation pairs. |
| Outcome: | The proposed model can detect missing and wrong translations in 15000 Chinese-English translation pairs. |
Machine Translation Hallucination Detection for Low and High Resource Languages using Large Language Models (2024.findings-emnlp)
Copied to clipboard
Kenza Benkirane, Laura Gongas, Shahar Pelles, Naomi Fuchs, Joshua Darmon, Pontus Stenetorp, David Adelani, Eduardo Sánchez
| Challenge: | Existing methods for detecting hallucinations in machine translation are limited for low-resource languages. |
| Approach: | They evaluate sentence-level hallucination detection approaches using Large Language Models (LLMs) they find that the choice of model is essential for performance. |
| Outcome: | The proposed models outperform the existing models in HRLs and LRLs on average by 0.16 MCC. |
Contextual Refinement of Translations: Large Language Models for Sentence and Document-Level Post-Editing (2024.naacl-long)
Copied to clipboard
| Challenge: | Large language models have demonstrated considerable success in various natural language processing tasks, but their performance in NMT tasks is still underexplored. |
| Approach: | They propose to use LLMs as automatic post-editors rather than direct translators to improve BLEU and COMET performance. |
| Outcome: | The proposed approach improves BLEU but COMET performance compared to in-context learning. |
Predicting Human Translation Difficulty Using Automatic Word Alignment (2023.findings-acl)
Copied to clipboard
| Challenge: | Translation difficulty is a problem when translators are required to resolve translation ambiguity from multiple possible translations. |
| Approach: | They use word alignments computed over large scale bilingual corpora to develop predictors of lexical translation difficulty. |
| Outcome: | The proposed method improves on a previous embedding-based approach and can contribute to a deeper understanding of cross-lingual differences and of causes of translation difficulty. |
Cheating to Identify Hard Problems for Neural Machine Translation (2023.findings-eacl)
Copied to clipboard
| Challenge: | We find that the most frequent tokens are not necessarily the most accurately translated due to these often being function words and punctuation that can be used more flexibly in translation. |
| Approach: | They propose a method to provide a compressed representation of the target as an input and a second method to fine-tune a standard transformer model. |
| Outcome: | The proposed method produces models with higher scores, but with more errors . the proposed method is based on two forms of cheating: providing a compressed representation of the target as an input and fine-tuning on the test set. |
Salute the Classic: Revisiting Challenges of Machine Translation in the Age of Large Language Models (2025.tacl-1)
Copied to clipboard
| Challenge: | a recent study revisits six core challenges that have influenced the evolution of Neural Machine Translation (NMT) domain mismatch, amount of parallel data, rare word prediction, translation of long sentences and sub-optimal beam search remain challenges in LLMs. |
| Approach: | They revisit core challenges that have acted as benchmarks for progress in NMT . they propose to revisit these challenges and offer insights into their relevance . |
| Outcome: | The proposed models significantly improve translation of sentences containing approximately 80 words, even translating documents up to 512 words. |
An Exploratory Analysis of Multilingual Word-Level Quality Estimation with Cross-Lingual Transformers (2021.acl-short)
Copied to clipboard
| Challenge: | Existing word-level quality estimation models require labelled data for each language pair and expensive maintenance. |
| Approach: | They propose to use multilingual QE models to generalise across languages . they propose to train models on other language pairs to predict word-level quality . |
| Outcome: | The proposed models generalise well across languages, making them more useful in real-world scenarios. |