Papers by Lucia Specia

Choosing What to Mask: More Informed Masking for Multimodal Machine Translation (2023.acl-srw)

Challenge:	a large-scale multimodal and multilingual dataset is used to facilitate research on visual grounding of words to images in their contextual usage in language.
Approach:	They propose a large-scale multimodal and multilingual dataset that aims to facilitate research on grounding words to images in their contextual usage in language.
Outcome:	The proposed dataset will facilitate research on visual grounding of words in their contextual usage in language.

Copied to clipboard

Julia Sato, Helena Caseli, Lucia Specia

Improving Neural Machine Translation Robustness via Data Augmentation: Beyond Back-Translation (D19-55)

Challenge:	Pre-trained language models have achieved remarkable results on several NLP tasks.
Approach:	They propose three new masking strategies for cross-lingual visual pre-training that focus on learning different linguistic patterns.
Outcome:	The proposed methods outperform the baseline model and achieve state-of-the-art accuracy on the Portuguese-English MMT task.

Copied to clipboard

Zhenhao Li, Lucia Specia

Responsible & Trustworthy NLP

Challenge:	Neural Machine Translation models are sensitive to noise in the input data.
Approach:	They propose new methods to extend limited noisy data and further improve NMT robustness to noise while keeping the models small.
Outcome:	The proposed methods extend limited noisy data and improve robustness to noise while keeping the models small.

Classification-based Quality Estimation: Small and Efficient Models for Real-world Applications (2021.emnlp-main)

Copied to clipboard

Shuo Sun, Ahmed El-Kishky, Vishrav Chaudhary, James Cross, Lucia Specia, Francisco Guzmán

MLQE-PE: A Multilingual Quality Estimation and Post-Editing Dataset (2022.lrec-1)

Challenge:	Sentence-level Quality estimation (QE) is traditionally a regression task . but large multilingual contextualized language models are expensive and infeasible for real-world applications.
Approach:	They evaluate several model compression techniques for QE and find they are inefficient . they argue that a full model parameterization is required to achieve SoTA results .
Outcome:	The proposed models are poorly expressive in a regression task, the authors argue . they show that reframing QE as a classification problem and evaluating models would improve their performance in real-world applications.

Copied to clipboard

Marina Fomicheva, Shuo Sun, Erick Fonseca, Chrysoula Zerva, Frédéric Blain, Vishrav Chaudhary, Francisco Guzmán, Nina Lopatina, Lucia Specia, André F. T. Martins

deepQuest: A Framework for Neural-based Quality Estimation (C18-1)

Challenge:	Existing datasets for machine translation quality estimation and post-editing have several shortcomings.
Approach:	They propose a dataset for machine translation quality estimation and automatic post-editing . they report the performance of baseline systems trained on the MLQE-PE dataset .
Outcome:	The proposed dataset contains human labels for up to 10,000 translations per language pair.

Copied to clipboard

Julia Ive, Frédéric Blain, Lucia Specia

A Post-Editing Dataset in the Legal Domain: Do we Underestimate Neural Machine Translation Quality? (2020.lrec-1)

Challenge:	Predicting Machine Translation (MT) quality has been limited to word and sentence-level prediction.
Approach:	They propose a framework that can generalize neural QE approaches to the level of documents.
Outcome:	The proposed framework outperforms state-of-the-art approaches on document-level quality estimates and is 40 times faster to train.

Copied to clipboard

Julia Ive, Lucia Specia, Sara Szoc, Tom Vanallemeersch, Joachim Van den Bogaert, Eduardo Farah, Christine Maroti, Artur Ventura, Maxim Khalilov

Simultaneous Machine Translation with Visual Context (2020.emnlp-main)

Challenge:	Current state-of-the-art in Neural Machine Translation (NMT) has reached remarkable progress, but human evaluations are often judged as having lower quality than top NMT systems.
Approach:	They propose to use a machine translation dataset with post-edited high-quality neural machine translation and independent human references to compare the results.
Outcome:	The proposed dataset includes 31K tuples including a source sentence, the respective machine translation by a neural machine translation system, and a post-edited version of such translation by professional translator.

Copied to clipboard

Ozan Caglayan, Julia Ive, Veneta Haralampieva, Pranava Madhyastha, Loïc Barrault, Lucia Specia

DiffuseDef: Improved Robustness to Adversarial Attacks via Iterative Denoising (2025.acl-long)

Challenge:	Simultaneous machine translation (SiMT) aims to reproduce human interpretation, where an interpreter translates spoken utterances as they are produced.
Approach:	They propose to add visual context to siMT to compensate for the missing source context . they show visual-grounded models are much better than commonly used global features .
Outcome:	The proposed models reach up to 3 BLEU points improvement under low latency scenarios.

Copied to clipboard

Zhenhao Li, Huichi Zhou, Marek Rei, Lucia Specia

Robustness in NLP

Challenge:	Existing adversarial defense methods for natural language processing still pose challenges to adversarials.
Approach:	They propose a novel adversarial defense method that incorporates a diffusion layer as a denoiser between the encoder and the classifier.
Outcome:	The proposed method improves over existing adversarial defense methods and achieves state-of-the-art performance against black-box and white-box adversarials.

Deep Copycat Networks for Text-to-Text Generation (D19-1)

Copied to clipboard

Julia Ive, Pranava Madhyastha, Lucia Specia

ASSET: A Dataset for Tuning and Evaluation of Sentence Simplification Models with Multiple Rewriting Transformations (2020.acl-main)

Challenge:	Text-to-text generation tasks require copying words from the input to the output.
Approach:	They propose a transformer-based pointer network for text-to-text generation which generates more abstractive summaries and a further extension of this architecture for automatic post-editing.
Outcome:	The proposed model outperforms existing models in text-to-text generation tasks and improves translation accuracy.

Copied to clipboard

Fernando Alva-Manchego, Louis Martin, Antoine Bordes, Carolina Scarton, Benoît Sagot, Lucia Specia

Are we Estimating or Guesstimating Translation Quality? (2020.acl-main)

Challenge:	Existing models for sentence simplification are focused on a single transformation, such as lexical paraphrasing or splitting.
Approach:	They propose a dataset for assessing sentence simplification in English using a crowdsourced multi-reference corpus.
Outcome:	The proposed dataset shows that it captures characteristics of simplicity better than other datasets.

Copied to clipboard

Shuo Sun, Francisco Guzmán, Lucia Specia

Translation Error Detection as Rationale Extraction (2022.findings-acl)

Challenge:	A carefully engineered ensemble of pre-trained multilingual language models won the QE shared task at WMT19.
Approach:	They propose to use pre-trained multilingual language models to train quality estimation for machine translation.
Outcome:	A carefully engineered ensemble of pre-trained language models wins the QE shared task at WMT19.

Copied to clipboard

Marina Fomicheva, Lucia Specia, Nikolaos Aletras

Exploiting Multimodal Reinforcement Learning for Simultaneous Machine Translation (2021.eacl-main)

Challenge:	Recent Quality Estimation models rely on translation errors to predict overall sentence quality, but detecting specific errors is a more challenging task.
Approach:	They propose to use a semi-supervised method to detect translation errors by attribution of relevance scores to inputs to explain model predictions.
Outcome:	The proposed method can detect translation errors and is compared with human models using a set of feature attribution methods.

Copied to clipboard

Julia Ive, Andy Mingren Li, Yishu Miao, Ozan Caglayan, Pranava Madhyastha, Lucia Specia

Guiding Visual Question Generation (2022.naacl-main)

Challenge:	Existing studies on multimodality in simultaneous machine translation have highlighted the challenges for the agent to maintain good translation quality while learning an optimal translation path.
Approach:	They propose a multimodal approach to simultaneous machine translation using reinforcement learning with strategies to integrate visual and textual information in both the agent and the environment.
Outcome:	The proposed multimodal approach improves translation quality while keeping latency low while providing visual cues.

Copied to clipboard

Nihir Vedd, Zixu Wang, Marek Rei, Yishu Miao, Lucia Specia

Visual Data in NLP

Challenge:	Existing approaches to Visual Question Generation (VQG) are trained to mimic an arbitrary choice of concept but only one or a few are captured by the human references.
Approach:	They propose a variant of Visual Question Generation which conditions the question generator on categorical information based on expectations on the type of question and the objects it should explore.
Outcome:	The proposed model improves on the current state of the art on an answer-category augmented VQA dataset and human evaluation validates that guidance helps the generation of questions that are grammatically coherent and relevant to the given image and objects.

Classifying Social Media Users before and after Depression Diagnosis via Their Language Usage: A Dataset and Study (2024.lrec-main)

Copied to clipboard

Falwah Alhamed, Julia Ive, Lucia Specia

An Exploratory Study on Multilingual Quality Estimation (2020.aacl-main)

Challenge:	Mental illness can negatively impact individuals’ quality of life as it is considered one of the causes of years lived with disability and it is related to high suicide rates.
Approach:	They collect first dataset of textual posts by same users before and after being diagnosed with depression and build multiple predictive models based on Transformers and BERT.
Outcome:	The proposed model can be used to detect depression and suicidal thoughts in users who are not diagnosed with depression or suicide.

Copied to clipboard

Shuo Sun, Marina Fomicheva, Frédéric Blain, Vishrav Chaudhary, Ahmed El-Kishky, Adithya Renduchintala, Francisco Guzmán, Lucia Specia

Bias Mitigation in Machine Translation Quality Estimation (2022.acl-long)

Challenge:	Existing approaches to predict the quality of machine translation use language-specific models, but they lack labelled data for each language pair.
Approach:	They propose to use scores from translation models to estimate quality of machine translations by predicting the quality of a translation at test time.
Outcome:	The proposed models outperform single-language models in less balanced quality label distributions and low-resource settings.

Copied to clipboard

Hanna Behnke, Marina Fomicheva, Lucia Specia

A Generative Framework for Simultaneous Machine Translation (2021.emnlp-main)

Challenge:	despite advances in machine translation, the accuracy and fluency of translations cannot be guaranteed without a reference translation.
Approach:	They propose to use auxiliary tasks to mitigate partial input bias . they aim to train a multitask architecture with an auxiliary binary classification task .
Outcome:	The proposed models reduce partial input bias while maintaining the overall performance.

Copied to clipboard

Yishu Miao, Phil Blunsom, Lucia Specia

Vis-Eval Metric Viewer: A Visualisation Tool for Inspecting and Evaluating Metric Scores of Machine Translation Output (N18-5)

Challenge:	Existing approaches use a fixed number of source words to translate or learn dynamic policies for the number of sources by reinforcement learning.
Approach:	They propose a generative framework that uses a latent variable to model read or translate actions at every time step and integrates out to consider all possible translation policies.
Outcome:	The proposed framework achieves the best BLEU scores on benchmark datasets.

Copied to clipboard

David Steele, Lucia Specia

Probing the Need for Visual Context in Multimodal Machine Translation (N19-1)

Challenge:	Many metrics have been proposed for Machine Translation (MT) that compare system translations against human references.
Approach:	They propose to use BLEU and METEOR to evaluate machine translations against human translations.
Outcome:	VisEval Metric Viewer (VEMV) provides visualisation of multiple evaluation scores so they can be easily interpreted by a user.

Copied to clipboard

Ozan Caglayan, Pranava Madhyastha, Lucia Specia, Loïc Barrault

Text Simplification from Professionally Produced Corpora (L18-1)

Challenge:	Current work on multimodal machine translation (MMT) suggests that the visual modality is either unnecessary or only marginally beneficial.
Approach:	They propose to use the visual modality to combine visual and textual information to generate better translations by partially depriving models from source-side textual context.
Outcome:	The proposed model can combine visual and textual information to generate better translations under limited textual context.

Copied to clipboard

Carolina Scarton, Gustavo Paetzold, Lucia Specia

Learning Simplifications for Specific Target Audiences (P18-2)

Challenge:	Existing approaches to Text Simplification rely on the Wikipedia-Simple Wikipedia parallel corpus, which is used for many tasks.
Approach:	They propose to use the Newsela corpus to extract 550, 644 complex-simple sentence pairs from the corpus and introduce a lexical simplifier that uses the corpu to generate candidate simplifications.
Outcome:	The proposed model outperforms state-of-the-art approaches and generates candidate simplifications from the newsela corpus.

Copied to clipboard

Carolina Scarton, Lucia Specia

Multimodal Conversation Modelling for Topic Derailment Detection (2022.findings-emnlp)

Challenge:	Text simplification is a monolingual text-to-text transformation task . data from TS data can contain multiple simplifications of the same original text .
Approach:	They propose to use sequence-to-sequence neural models to build models tailored for specific grade levels.
Outcome:	The proposed model outperforms state-of-the-art approaches for a monolingual text-to-text transformation task.

Copied to clipboard

Zhenhao Li, Marek Rei, Lucia Specia

Visual Cues and Error Correction for Translation Robustness (2021.findings-emnlp)

Challenge:	Existing work on analysing textual dialogues that derailed into toxic content ignores visual information, such as images and videos.
Approach:	They propose a new multimodal conversational architecture that utilises visual and conversational contexts to classify comments for derailment.
Outcome:	The proposed approach outperforms existing methods and is more robust to textual noise.

Copied to clipboard

Zhenhao Li, Marek Rei, Lucia Specia

Continual Quality Estimation with Online Bayesian Meta-Learning (2021.acl-short)

Challenge:	Existing robustness techniques fail when faced with unseen types of noise and their performance degrades on clean texts.
Approach:	They propose visual context to improve translation robustness for noisy texts . they also propose an error correction training regime that can be used as an auxiliary task .
Outcome:	The proposed training regime improves translation robustness on noisy texts while maintaining translation quality on clean texts.

Copied to clipboard

Abiola Obamuyide, Marina Fomicheva, Lucia Specia

Unsupervised Quality Estimation for Neural Machine Translation (2020.tacl-1)

Challenge:	Existing quality estimation models for machine translation are trained and evaluated in a static setting . however, in real-life settings, test data may differ from training data .
Approach:	They propose an online Bayesian meta-learning framework for continuous training of QE models that adapts to the needs of different users while being robust to distributional shifts in training and test data.
Outcome:	The proposed framework adapts to the needs of different users while being robust to distributional shifts in training and test data.

Copied to clipboard

Marina Fomicheva, Shuo Sun, Lisa Yankovskaya, Frédéric Blain, Francisco Guzmán, Mark Fishel, Nikolaos Aletras, Vishrav Chaudhary, Lucia Specia

Defoiling Foiled Image Captions (N18-2)

Challenge:	Existing approaches require large amounts of expert annotated data, computation, and time for training.
Approach:	They propose an unsupervised approach to QE where no training is required . they use a dataset that enables work on both black-box and glass-box approaches .
Outcome:	The proposed approach rivals state-of-the-art supervised QE models in terms of correlation with human judgments of quality.

Copied to clipboard

Pranava Swaroop Madhyastha, Josiah Wang, Lucia Specia

Leveraging Pre-trained Language Models for Gender Debiasing (2022.lrec-1)

Challenge:	Existing models for vision-to-language tasks do not understand images sufficiently, despite their impressive performance.
Approach:	They propose to use explicit object information to solve foiled caption detection problem . they propose to replace a word in caption with a semantically similar word .
Outcome:	The proposed model achieves state-of-the-art on a recently published dataset with scores exceeding those achieved by humans on the task.

Copied to clipboard

Nishtha Jain, Declan Groves, Lucia Specia, Maja Popović

Object Counts! Bringing Explicit Detections Back into Image Captioning (N18-1)

Challenge:	Existing methods to reduce gender bias in natural language are costly and time-consuming.
Approach:	They propose a method to generate gender variants for a given text using pre-trained language models as the resource without any task-specific labelled data.
Outcome:	The proposed method can reduce gender bias in a language generation context without a task-specific labelled data.

Copied to clipboard

Josiah Wang, Pranava Swaroop Madhyastha, Lucia Specia

SimPA: A Sentence-Level Simplification Corpus for the Public Administration Domain (L18-1)

Challenge:	Existing approaches to image captioning use explicit object detectors as an intermediate step, but they bypass the explicit detection phase and instead generate captions directly from image embeddings.
Approach:	They argue that explicit detections provide rich semantic information and can thus be used as an interpretable representation to better understand why end-to-end image captioning systems work well.
Outcome:	The proposed methods can be used to understand why end-to-end captioning systems work well.

Copied to clipboard

Carolina Scarton, Gustavo Paetzold, Lucia Specia

BERTGen: Multi-task Generation through BERT (2021.acl-long)

Challenge:	lexical simplification is the task of reducing lexically and/or structural complexity of texts.
Approach:	They propose to collect manual simplifications for 1,100 original sentences using a sentence-level corpus from the Public Administration domain.
Outcome:	The proposed corpus contains 1,100 original sentences with manual simplifications collected through a two-stage process.

Copied to clipboard

Faidon Mitzalis, Ozan Caglayan, Pranava Madhyastha, Lucia Specia

Knowledge Distillation for Quality Estimation (2021.findings-acl)

Challenge:	Recent work in unsupervised and self-supervised pre-training has revolutionised the field of natural language understanding (NLU).
Approach:	They propose to use multimodal and multilingual pre-trained models to extend BERT by fusing them together for language generation tasks.
Outcome:	The proposed model outperforms baseline models in image captioning, machine translation and multimodal machine translation tasks and is competitive with supervised counterparts.

Copied to clipboard

Amit Gajbhiye, Marina Fomicheva, Fernando Alva-Manchego, Frédéric Blain, Abiola Obamuyide, Nikolaos Aletras, Lucia Specia

Multimodal Quality Estimation for Machine Translation (2020.acl-main)

Challenge:	Recent success in Quality Estimation stems from the use of multilingual pre-trained models, where large models lead to impressive results.
Approach:	They propose to transfer knowledge from a strong QE teacher model to a much smaller model with a different, shallower architecture.
Outcome:	The proposed model performs better than distilled models with 8x fewer parameters.

Copied to clipboard

Shu Okabe, Frédéric Blain, Lucia Specia

Curious Case of Language Generation Evaluation Metrics: A Cautionary Tale (2020.coling-main)

Challenge:	Existing work has only explored textual context.
Approach:	They propose to use visual and text modalities to explore Quality Estimation for Machine Translation and integrate them into multimodal QE frameworks.
Outcome:	The proposed approaches improve on sentence-level and document-level predictions using visual features extracted from images.

Copied to clipboard

Ozan Caglayan, Pranava Madhyastha, Lucia Specia

Multimodal Lexical Translation (L18-1)

Challenge:	a few popular metrics are still used to evaluate language generation systems despite their known limitations.
Approach:	They propose to use automatic metrics to evaluate language generation systems . they show that they prefer system outputs to human-authored texts .
Outcome:	The proposed metrics are insensitive to correct translations of rare words and can yield high scores when given a single sentence as system output for the entire test set.

Copied to clipboard

Chiraag Lala, Lucia Specia

Uncertainty Aware Review Hallucination for Science Article Classification (2021.findings-acl)

Challenge:	Multimodal Lexical Translation (MLT) is a task that aims to translate ambiguous words given their context -an image and a sentence in the source language.
Approach:	They introduce a task to translate an ambiguous word given its context -an image and a sentence in the source language.
Outcome:	The proposed task is based on the Multi30K dataset and uses word-alignment followed by human inspection to select subsets of the dataset which are difficult to translate.

Copied to clipboard

Korbinian Friedl, Georgios Rizos, Lukas Stappen, Madina Hasan, Lucia Specia, Thomas Hain, Björn Schuller

Exploring Supervised and Unsupervised Rewards in Machine Translation (2021.eacl-main)

Challenge:	Existing approaches to peer review support are limited in their use of available information and subjectivity.
Approach:	They propose to use aleatory uncertainty and loss importance interpolations to model review representations at test time to provide a realistic evaluation framework.
Outcome:	The proposed framework makes better use of the available information and is realistic with respect to the limitations set by the task 1 .

Copied to clipboard

Julia Ive, Zixu Wang, Marina Fomicheva, Lucia Specia

Backtranslation Feedback Improves User Confidence in MT, Not Quality (2021.naacl-main)

Challenge:	Autoregressive sequence-to-sequence (seq2sequ) neural architectures have become the de facto approach in Machine Translation (MT).
Approach:	They propose to make models less reliant on cross-entropy loss and evaluation metrics . they propose an entropicity-regularised RL method that explores the action space .
Outcome:	The proposed method exploits the action space and unsupervised reward function to balance between exploration and exploitation.

Copied to clipboard

Vilém Zouhar, Michal Novák, Matúš Žilinec, Ondřej Bojar, Mateo Obregón, Robin L. Hill, Frédéric Blain, Marina Fomicheva, Lucia Specia, Lisa Yankovskaya

Cross-lingual Visual Pre-training for Multimodal Machine Translation (2021.eacl-main)

Challenge:	Inbound translation is a modern need for which the user experience has significant room for improvement, beyond the basic machine translation facility.
Approach:	They propose to provide cues that indicate the quality of MT output as well as suggest possible rephrasing of the source language.
Outcome:	The proposed feedback module increases user confidence in the produced translation, but not the objective quality.

Copied to clipboard

Ozan Caglayan, Menekse Kuyu, Mustafa Sercan Amac, Pranava Madhyastha, Erkut Erdem, Aykut Erdem, Lucia Specia

Distilling Translations with Visual Awareness (P19-1)

Challenge:	Pre-trained language models have been shown to improve performance in many natural language tasks.
Approach:	They propose to combine cross-lingual and visual pre-training to learn visually-grounded cross-linguistic representations using masked region classification and three-way parallel vision & language corpora.
Outcome:	The proposed models obtain state-of-the-art performance when fine-tuned for multimodal machine translation.

Copied to clipboard

Julia Ive, Pranava Madhyastha, Lucia Specia

deepQuest-py: Large and Distilled Models for Quality Estimation (2021.emnlp-demo)

Challenge:	Existing work on multimodal machine translation has shown that visual information is only needed in very specific cases, for example in the presence of ambiguous words where the textual context is not sufficient.
Approach:	They propose a translate-and-refine approach to multimodal machine translation where images are only used by a second stage decoder to generate a good first draft translation and to improve over this draft.
Outcome:	The proposed approach generates a good translation and improves over the draft by making better use of the target language textual context and making use of visual context.

Copied to clipboard

Fernando Alva-Manchego, Abiola Obamuyide, Amit Gajbhiye, Frédéric Blain, Marina Fomicheva, Lucia Specia

Multilingual and Multimodal Learning for Brazilian Portuguese (2022.lrec-1)

Challenge:	Quality Estimation (QE) is a tool for machine translation that predicts how good translations are without comparing them to gold-standard references.
Approach:	They introduce a framework for training and evaluation of large and light-weight models for Quality Estimation (QE) they use pre-trained Transformers to train large and efficient QE models.
Outcome:	The framework provides access to state-of-the-art models based on pre-trained Transformers for sentence-level and word-level QE and a web interface for testing and visualising their predictions.

Copied to clipboard

Júlia Sato, Helena Caseli, Lucia Specia

EASSE: Easier Automatic Sentence Simplification Evaluation (D19-3)

Challenge:	Existing models that learn multimodal and multilingual representations perform better in many natural language tasks.
Approach:	They use a multimodal and multilingual corpus to test its generalization ability for other languages . they achieve a BLEU score of 51.8 and a METEOR score of 78.0 on the test set .
Outcome:	The proposed model outperforms the existing model on a Portuguese-English multimodal translation task.

Copied to clipboard

Fernando Alva-Manchego, Louis Martin, Carolina Scarton, Lucia Specia

FIND: Human-in-the-Loop Debugging Deep Text Classifiers (2020.emnlp-main)

Challenge:	EASSE provides access to a broad range of evaluation resources including standard automatic metrics, word-level accuracy scores and reference-independent quality estimation features.
Approach:	They propose to provide a Python package that provides access to automatic evaluation and comparison of Sentence Simplification (SS) systems.
Outcome:	The proposed tool allows comparison and understanding of the performance of Sentence Simplification (SS) systems.

Copied to clipboard

Piyawat Lertvittayakumjorn, Lucia Specia, Francesca Toni

Multi-Hypothesis Machine Translation Evaluation (2020.acl-main)

Challenge:	Existing models are limited in the number of available datasets and lack the necessary tools to improve them.
Approach:	They propose a framework which enables humans to debug deep learning text classifiers by disabling irrelevant hidden features.
Outcome:	Experiments show that using FIND, humans can improve CNN text classifiers trained on different types of imperfect datasets.

Copied to clipboard

Marina Fomicheva, Lucia Specia, Francisco Guzmán

SentSim: Crosslingual Semantic Evaluation of Machine Translation (2021.naacl-main)

Challenge:	Reliably evaluating Machine Translation (MT) through automated metrics is a long-standing problem.
Approach:	They propose to use MT models to generate multiple diverse translations and use them as surrogates to reference translations to obtain a quantification of translation variability.
Outcome:	The proposed approach improves correlation with human judgements of quality by 15%.

Copied to clipboard

Yurun Song, Junchen Zhao, Lucia Specia

Quality Estimation without Human-labeled Data (2021.eacl-main)

Challenge:	Machine translation (MT) is currently evaluated in one of two ways: monolingually or trained crosslingually by building a supervised model to predict quality scores from human-labeled data.
Approach:	They propose an unsupervised model that directly compares the source and machine translated sentence using strong pretrained multilingual word and sentence representations.
Outcome:	The proposed model outperforms glass-box approaches to quality estimation that rely on a supervised model.

Copied to clipboard

Yi-Lin Tuan, Ahmed El-Kishky, Adithya Renduchintala, Vishrav Chaudhary, Francisco Guzmán, Lucia Specia