On Systematic Style Differences between Unsupervised and Supervised MT and an Application for High-Resource Machine Translation (2022.naacl-main)
Copied to clipboard
| Challenge: | Modern unsupervised machine translation systems reach reasonable translation quality under clean and controlled data conditions. |
| Approach: | They compare unsupervised and supervised machine translation systems of similar quality . they combine the benefits of both methods into a single system . |
| Outcome: | The proposed system improves adequacy and fluency as measured by human evaluators. |
Similar Papers
Unsupervised Machine Translation in Real-World Scenarios (2022.lrec-1)
Copied to clipboard
Ona de Gibert Bonet, Iakes Goenaga, Jordi Armengol-Estapé, Olatz Perez-de-Viñaspre, Carla Parra Escartín, Marina Sanchez, Mārcis Pinnis, Gorka Labaka, Maite Melero
| Challenge: | a recent study has shown that unsupervised methods rely on monolingual corpora to build MT systems. |
| Approach: | They present the results of the MT4All CEF project using monolingual corpora . they propose to generate bilingual dictionaries and translation models from monolingual data . |
| Outcome: | The proposed method generates bilingual dictionaries and translation models from monolingual corpora . results show that it is comparable to general domain supervised translation . |
UScore: An Effective Approach to Fully Unsupervised Evaluation Metrics for Machine Translation (2023.eacl-main)
Copied to clipboard
| Challenge: | supervised evaluation metrics are not available for machine translation, despite their wide dissemination. |
| Approach: | They develop fully unsupervised evaluation metrics that leverage parallel data and evaluation metric induction. |
| Outcome: | The proposed metrics beat supervised competitors on 4 out of 5 evaluation datasets. |
On the Limitations of Unsupervised Bilingual Dictionary Induction (P18-1)
Copied to clipboard
| Challenge: | Unsupervised machine translation does not require cross-lingual supervision, whether a dictionary, translations, or comparable corpora. |
| Approach: | They propose an adversarial, unsupervised cross-lingual word embedding technique for bilingual dictionary induction that exploits a weak supervision signal from identical words. |
| Outcome: | The proposed model relies heavily on an adversarial, unsupervised cross-lingual word embedding technique for bilingual dictionary induction. |
SentSim: Crosslingual Semantic Evaluation of Machine Translation (2021.naacl-main)
Copied to clipboard
| Challenge: | Machine translation (MT) is currently evaluated in one of two ways: monolingually or trained crosslingually by building a supervised model to predict quality scores from human-labeled data. |
| Approach: | They propose an unsupervised model that directly compares the source and machine translated sentence using strong pretrained multilingual word and sentence representations. |
| Outcome: | The proposed model outperforms glass-box approaches to quality estimation that rely on a supervised model. |
Harnessing Multilinguality in Unsupervised Machine Translation for Rare Languages (2021.naacl-main)
Copied to clipboard
| Challenge: | Unsupervised translation systems have impressive performance on resource-rich language pairs . however, in more realistic settings, unsupervised systems perform poorly . |
| Approach: | They propose a model for 5 low-resource languages that leverages monolingual and auxiliary parallel data from other high-resourced languages. |
| Outcome: | The proposed model outperforms state-of-the-art models on low-resource languages . it also matches the current state- of-the art model for Nepali-English . |
Has Machine Translation Evaluation Achieved Human Parity? The Human Reference and the Limits of Progress (2025.acl-short)
Copied to clipboard
| Challenge: | In machine translation evaluation, metric performance is assessed based on agreement with human judgments. |
| Approach: | They incorporate human baselines into the MT meta-evaluation to gain a clearer understanding of metric performance and establish an upper bound. |
| Outcome: | The results suggest human parity, but there are several reasons to caution . |
Revisiting Machine Translation for Cross-lingual Classification (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent work in cross-lingual learning has pivoted around multilingual models, which are typically pretrained on unlabeled corpora in multiple languages using some form of language modeling objective. |
| Approach: | They propose to use a stronger machine translation system to mitigat mismatch between training on original text and running inference on machine translated text. |
| Outcome: | The proposed approach is highly task dependent and calls into question the dominance of multilingual models for cross-lingual classification. |
Evaluating Automatic Metrics with Incremental Machine Translation Systems (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies have shown that neural metrics are more reliable than non-neural metrics. |
| Approach: | They propose to use commercial machine translations to evaluate machine translation metrics based on their preference for more recent outputs. |
| Outcome: | The proposed dataset confirms several previous findings, including the advantage of neural metrics over non-neural ones, and also explores the debated issue of how MT quality affects metric reliability. |
Scientific Credibility of Machine Translation Research: A Meta-Evaluation of 769 Papers (2021.acl-long)
Copied to clipboard
| Challenge: | a meta-evaluation of machine translation (MT) has been conducted in 769 research papers . a recent study shows that evaluation practices have changed over the past decade . |
| Approach: | They propose a meta-evaluation method for machine translation that uses BLEU scores to evaluate MT performance. |
| Outcome: | The proposed meta-evaluation of machine translation shows that evaluation practices have changed over the past decade . the authors suggest that the evaluation process should be streamlined and standardized to ensure the validity of the evaluation method . |
Tangled up in BLEU: Reevaluating the Evaluation of Automatic Machine Translation Evaluation Metrics (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods for judging metrics are sensitive to the translations used for evaluation, leading to falsely confident conclusions about a metric’s efficacy. |
| Approach: | They propose a method for thresholding performance improvement under an automatic metric against human judgements by using a pairwise system ranking method. |
| Outcome: | The proposed method allows quantification of type I versus type II errors incurred, i.e., insignificant human differences in system quality that are accepted, and significant human differences that are rejected. |