Papers by Marina Fomicheva
MLQE-PE: A Multilingual Quality Estimation and Post-Editing Dataset (2022.lrec-1)
Copied to clipboard
Marina Fomicheva, Shuo Sun, Erick Fonseca, Chrysoula Zerva, Frédéric Blain, Vishrav Chaudhary, Francisco Guzmán, Nina Lopatina, Lucia Specia, André F. T. Martins
| Challenge: | Existing datasets for machine translation quality estimation and post-editing have several shortcomings. |
| Approach: | They propose a dataset for machine translation quality estimation and automatic post-editing . they report the performance of baseline systems trained on the MLQE-PE dataset . |
| Outcome: | The proposed dataset contains human labels for up to 10,000 translations per language pair. |
Translation Error Detection as Rationale Extraction (2022.findings-acl)
Copied to clipboard
| Challenge: | Recent Quality Estimation models rely on translation errors to predict overall sentence quality, but detecting specific errors is a more challenging task. |
| Approach: | They propose to use a semi-supervised method to detect translation errors by attribution of relevance scores to inputs to explain model predictions. |
| Outcome: | The proposed method can detect translation errors and is compared with human models using a set of feature attribution methods. |
An Exploratory Study on Multilingual Quality Estimation (2020.aacl-main)
Copied to clipboard
Shuo Sun, Marina Fomicheva, Frédéric Blain, Vishrav Chaudhary, Ahmed El-Kishky, Adithya Renduchintala, Francisco Guzmán, Lucia Specia
| Challenge: | Existing approaches to predict the quality of machine translation use language-specific models, but they lack labelled data for each language pair. |
| Approach: | They propose to use scores from translation models to estimate quality of machine translations by predicting the quality of a translation at test time. |
| Outcome: | The proposed models outperform single-language models in less balanced quality label distributions and low-resource settings. |
Bias Mitigation in Machine Translation Quality Estimation (2022.acl-long)
Copied to clipboard
| Challenge: | despite advances in machine translation, the accuracy and fluency of translations cannot be guaranteed without a reference translation. |
| Approach: | They propose to use auxiliary tasks to mitigate partial input bias . they aim to train a multitask architecture with an auxiliary binary classification task . |
| Outcome: | The proposed models reduce partial input bias while maintaining the overall performance. |
Continual Quality Estimation with Online Bayesian Meta-Learning (2021.acl-short)
Copied to clipboard
| Challenge: | Existing quality estimation models for machine translation are trained and evaluated in a static setting . however, in real-life settings, test data may differ from training data . |
| Approach: | They propose an online Bayesian meta-learning framework for continuous training of QE models that adapts to the needs of different users while being robust to distributional shifts in training and test data. |
| Outcome: | The proposed framework adapts to the needs of different users while being robust to distributional shifts in training and test data. |
Unsupervised Quality Estimation for Neural Machine Translation (2020.tacl-1)
Copied to clipboard
Marina Fomicheva, Shuo Sun, Lisa Yankovskaya, Frédéric Blain, Francisco Guzmán, Mark Fishel, Nikolaos Aletras, Vishrav Chaudhary, Lucia Specia
| Challenge: | Existing approaches require large amounts of expert annotated data, computation, and time for training. |
| Approach: | They propose an unsupervised approach to QE where no training is required . they use a dataset that enables work on both black-box and glass-box approaches . |
| Outcome: | The proposed approach rivals state-of-the-art supervised QE models in terms of correlation with human judgments of quality. |
Knowledge Distillation for Quality Estimation (2021.findings-acl)
Copied to clipboard
Amit Gajbhiye, Marina Fomicheva, Fernando Alva-Manchego, Frédéric Blain, Abiola Obamuyide, Nikolaos Aletras, Lucia Specia
| Challenge: | Recent success in Quality Estimation stems from the use of multilingual pre-trained models, where large models lead to impressive results. |
| Approach: | They propose to transfer knowledge from a strong QE teacher model to a much smaller model with a different, shallower architecture. |
| Outcome: | The proposed model performs better than distilled models with 8x fewer parameters. |
Exploring Supervised and Unsupervised Rewards in Machine Translation (2021.eacl-main)
Copied to clipboard
| Challenge: | Autoregressive sequence-to-sequence (seq2sequ) neural architectures have become the de facto approach in Machine Translation (MT). |
| Approach: | They propose to make models less reliant on cross-entropy loss and evaluation metrics . they propose an entropicity-regularised RL method that explores the action space . |
| Outcome: | The proposed method exploits the action space and unsupervised reward function to balance between exploration and exploitation. |
Backtranslation Feedback Improves User Confidence in MT, Not Quality (2021.naacl-main)
Copied to clipboard
Vilém Zouhar, Michal Novák, Matúš Žilinec, Ondřej Bojar, Mateo Obregón, Robin L. Hill, Frédéric Blain, Marina Fomicheva, Lucia Specia, Lisa Yankovskaya
| Challenge: | Inbound translation is a modern need for which the user experience has significant room for improvement, beyond the basic machine translation facility. |
| Approach: | They propose to provide cues that indicate the quality of MT output as well as suggest possible rephrasing of the source language. |
| Outcome: | The proposed feedback module increases user confidence in the produced translation, but not the objective quality. |
deepQuest-py: Large and Distilled Models for Quality Estimation (2021.emnlp-demo)
Copied to clipboard
Fernando Alva-Manchego, Abiola Obamuyide, Amit Gajbhiye, Frédéric Blain, Marina Fomicheva, Lucia Specia
| Challenge: | Quality Estimation (QE) is a tool for machine translation that predicts how good translations are without comparing them to gold-standard references. |
| Approach: | They introduce a framework for training and evaluation of large and light-weight models for Quality Estimation (QE) they use pre-trained Transformers to train large and efficient QE models. |
| Outcome: | The framework provides access to state-of-the-art models based on pre-trained Transformers for sentence-level and word-level QE and a web interface for testing and visualising their predictions. |
Multi-Hypothesis Machine Translation Evaluation (2020.acl-main)
Copied to clipboard
| Challenge: | Reliably evaluating Machine Translation (MT) through automated metrics is a long-standing problem. |
| Approach: | They propose to use MT models to generate multiple diverse translations and use them as surrogates to reference translations to obtain a quantification of translation variability. |
| Outcome: | The proposed approach improves correlation with human judgements of quality by 15%. |