| Challenge: | Noisy input text can cause disastrous mistranslations in most modern machine translation systems. |
| Approach: | They propose a benchmark dataset for Machine Translation of Noisy Text (MTNT) they use reddit comments and professionally sourced translations to examine noise types. |
| Outcome: | The proposed dataset can provide an attractive testbed for noise-robust machine translation systems. |
Similar Papers
Ask Language Model to Clean Your Noisy Translation Data (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Neural machine translation models exhibit a noticeable decline in translation quality when exposed to noisy input. |
| Approach: | They use a dataset to evaluate the robustness of NMT models against noisy inputs. |
| Outcome: | The proposed dataset cleaners the noise from the target sentences while preserving the semantic integrity of the original sentences. |
Improving Robustness of Machine Translation with Synthetic Noise (N19-1)
Copied to clipboard
| Challenge: | Recent work on MT robustness has demonstrated the need to build or adapt systems that are resilient to such noise. |
| Approach: | They propose to synthesize natural noise in social media data to enhance robustness of MT systems by leveraging natural noise. |
| Outcome: | The proposed method can make a vanilla MT system more resilient to noise, partially mitigating loss in accuracy resulting therefrom. |
An Empirical Study on the Robustness of Massively Multilingual Neural Machine Translation (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent years have witnessed that massively multilingual neural machine translation (MMNMT) achieves a remarkable progress in both high- and low-resource language translation. |
| Approach: | They propose to use a robustness evaluation benchmark dataset to assess the translation robustness of Indonesian-Chinese translation in the face of various naturally occurring noise. |
| Outcome: | The proposed dataset is publicly available at https://github.com/ID-ZH-MTRobustEval. |
Did Translation Models Get More Robust Without Anyone Even Noticing? (2025.acl-long)
Copied to clipboard
| Challenge: | Neural machine translation models are highly sensitive to “noisy” inputs, such as spelling errors, abbreviations, and formatting issues. |
| Approach: | They revisit this insight in light of recent multilingual MT models and large language models applied to machine translation. |
| Outcome: | The proposed models perform better on clean data than previous models, but none of the open models use robustness techniques. |
Machine Translation of Restaurant Reviews: New Corpus for Domain Adaptation and Robustness (D19-56)
Copied to clipboard
Alexandre Berard, Ioan Calapodescu, Marc Dymetman, Claude Roux, Jean-Luc Meunier, Vassilina Nikoulina
| Challenge: | BLEU: MT is a very robust and efficient way to translate user-generated content. |
| Approach: | They propose a task to encourage research on MT robustness and domain adaptation . they ask professionals to translate 11.5k french 4SQ reviews to English . |
| Outcome: | The proposed task improves on the existing MT systems in a real-world scenario . the proposed methods improve translation accuracy and sentiment analysis . |
Scientific Credibility of Machine Translation Research: A Meta-Evaluation of 769 Papers (2021.acl-long)
Copied to clipboard
| Challenge: | a meta-evaluation of machine translation (MT) has been conducted in 769 research papers . a recent study shows that evaluation practices have changed over the past decade . |
| Approach: | They propose a meta-evaluation method for machine translation that uses BLEU scores to evaluate MT performance. |
| Outcome: | The proposed meta-evaluation of machine translation shows that evaluation practices have changed over the past decade . the authors suggest that the evaluation process should be streamlined and standardized to ensure the validity of the evaluation method . |
Upping the Ante: Towards a Better Benchmark for Chinese-to-English Machine Translation (L18-1)
Copied to clipboard
| Challenge: | Currently, there is no widely accepted standard for evaluation of machine translation (MT) for Chinese-to-English translation, there are no standard for standardized training sets, development sets, and test sets. |
| Approach: | They propose to use Chinese-to-English machine translation as a benchmark . they build a highly competitive state-of-the-art MT system that outperforms reported results . |
| Outcome: | The proposed system outperforms reported results on NIST OpenMT test sets in almost all papers published in major conferences and journals in computational linguistics and artificial intelligence in the past 11 years. |
mT5: A Massively Multilingual Pre-trained Text-to-Text Transformer (2021.naacl-main)
Copied to clipboard
Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, Colin Raffel
| Challenge: | Current natural language processing pipelines often use transfer learning, where a model is pre-trained on a data-rich task before being fine-tuned on . this significantly limits their use given that roughly 80% of the world population does not speak English. |
| Approach: | They introduce a multilingual variant of T5 that was pre-trained on a new Common Crawl-based dataset covering 101 languages. |
| Outcome: | The proposed model achieves state-of-the-art on multilingual benchmarks and a simple technique to prevent accidental translation in the zero-shot setting. |
Tilde MT Platform for Developing Client Specific MT Solutions (L18-1)
Copied to clipboard
| Challenge: | a growing demand for translations and multilingual content is surpassing the supply of professional translation services. |
| Approach: | They present a custom machine translation platform called Tilde MT that provides linguistic data storage, data cleaning and normalisation, statistical and neural machine translation system training and hosting functionality. |
| Outcome: | The proposed platform provides linguistic data storage, data cleaning and normalisation, statistical and neural machine translation system training and hosting functionality, and wide integration capabilities. |
CoCoA-MT: A Dataset and Benchmark for Contrastive Controlled MT with Application to Formality (2022.findings-naacl)
Copied to clipboard
| Challenge: | Specific problems arise when translating from English into languages with formality markers, such as “Are you sure?” . Using wrong or inconsistent tone may be perceived as inappropriate or jarring for users of certain cultures and demographics. |
| Approach: | They propose to train formality-controlled models by fine-tuning on labeled contrastive data and a metric to evaluate them. |
| Outcome: | The proposed model achieves high accuracy (82% in-domain and 73% out-of-domain) while maintaining overall quality. |