Papers by Sweta Agrawal
A Context-aware Framework for Translation-mediated Conversations (2026.tacl-1)
Copied to clipboard
| Challenge: | Existing systems that bridge language barriers can introduce errors leading to misunderstandings and conversation breakdown. |
| Approach: | They propose a framework to integrate contextual information into automatic translation systems . they validate the framework on customer chat and user-assistant interaction . |
| Outcome: | The proposed framework consistently produces better translations than state-of-the-art systems on two task-oriented domains. |
Controlling Text Complexity in Neural Machine Translation (D19-1)
Copied to clipboard
| Challenge: | Prior work on text complexity has focused on simplifying input text in one language, primarily English. |
| Approach: | They propose a method to align news articles written for different levels of target language proficiency. |
| Outcome: | The proposed model outperforms pipeline approaches that translate and simplify text independently. |
Understanding and Detecting Hallucinations in Neural Machine Translation via Model Introspection (2023.tacl-1)
Copied to clipboard
| Challenge: | Neural sequence generation models produce outputs that are unrelated to the source text, and are potentially harmful, yet it remains unclear in what conditions they arise and how to mitigate their impact. |
| Approach: | They first identify internal model symptoms of hallucinations by analyzing the relative token contributions to the generation in contrastive hallucinous vs. non-hallucinated outputs generated via source perturbations. |
| Outcome: | The proposed detector outperforms both baseline models and strong classifiers on English-Chinese and German-English translation test beds. |
Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets (2022.tacl-1)
Copied to clipboard
Julia Kreutzer, Isaac Caswell, Lisa Wang, Ahsan Wahab, Daan van Esch, Nasanbayar Ulzii-Orshikh, Allahsera Tapo, Nishant Subramani, Artem Sokolov, Claytone Sikasote, Monang Setyawan, Supheakmungkol Sarin, Sokhar Samb, Benoît Sagot, Clara Rivera, Annette Rios, Isabel Papadimitriou, Salomey Osei, Pedro Ortiz Suarez, Iroro Orife, Kelechi Ogueji, Andre Niyongabo Rubungo, Toan Q. Nguyen, Mathias Müller, André Müller, Shamsuddeen Hassan Muhammad, Nanda Muhammad, Ayanda Mnyakeni, Jamshidbek Mirzakhalov, Tapiwanashe Matangira, Colin Leong, Nze Lawson, Sneha Kudugunta, Yacine Jernite, Mathias Jenny, Orhan Firat, Bonaventure F. P. Dossou, Sakhile Dlamini, Nisansa de Silva, Sakine Çabuk Ballı, Stella Biderman, Alessia Battisti, Ahmed Baruwa, Ankur Bapna, Pallavi Baljekar, Israel Abebe Azime, Ayodele Awokoya, Duygu Ataman, Orevaoghene Ahia, Oghenefego Ahia, Sweta Agrawal, Mofetoluwa Adeyemi
| Challenge: | Lower-resource corpora have systematic issues, including mislabeled or nonstandard/ambiguous language codes. |
| Approach: | They manually audit the quality of 205 language-specific corpora released with five major public datasets. |
| Outcome: | The results show that lower-resource corpora have systematic issues even for non-proficient speakers. |
Evaluating the Evaluation Metrics for Style Transfer: A Case Study in Multilingual Formality Transfer (2021.emnlp-main)
Copied to clipboard
| Challenge: | a lack of standardized and reliable methods for automatic evaluation hinders ST . prior work has employed as many as nine different automatic systems to rate formality alone . |
| Approach: | They evaluate automatic metrics on the oft-researched task of formality style transfer . they outline best practices for automatic evaluation in (formality) style transfer and identify models that correlate well with human judgments. |
| Outcome: | The proposed models correlate well with human judgments and are robust across languages. |
Translate Smart, not Hard: Cascaded Translation Systems with Quality-Aware Deferral (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing quality estimation metrics are used to design effective deferral rules for machine translation. |
| Approach: | They propose a simple yet effective approach for machine translation using existing quality estimation metrics as deferral rules. |
| Outcome: | The proposed approach outperforms existing models in large translation tasks while reducing computational costs. |
Assessing Reference-Free Peer Evaluation for Machine Translation (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to evaluate machine translation output are based on comparing MT output to one or more reference translations. |
| Approach: | They propose to use probabilities given by a large, multilingual model as a reference-free metric. |
| Outcome: | The proposed model is robust and likely to offer reasonable performance across a broad spectrum of domains and different system qualities. |
An Imitation Learning Curriculum for Text Editing with Non-Autoregressive Models (2022.acl-long)
Copied to clipboard
| Challenge: | et al. (2017) show that imitation learning algorithms for machine translation introduce mismatches between training and inference that lead to undertraining and poor generalization in editing scenarios. |
| Approach: | They propose a framework for training non-autoregressive sequence-to-sequence models for editing tasks where the original input sequence is iteratively edited to produce the output. |
| Outcome: | The proposed framework significantly improves output quality and controls complexity better on the simplification task. |
A Non-Autoregressive Edit-Based Approach to Controllable Text Simplification (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing models that generate generic simplified outputs for a given source text have been used to specify output properties. |
| Approach: | They propose a non-autoregressive model that iteratively edits an input sequence and incorporates lexical complexity information into the refinement process to generate simplifications that better match the desired output complexity. |
| Outcome: | The proposed model incorporates lexical complexity information into the refinement process to achieve more complex simplification operations such as content deletion and paraphrasing, as well as sentence splitting. |
Fine-Grained Reward Optimization for Machine Translation using Error Severity Mappings (2026.tacl-1)
Copied to clipboard
Miguel Moura Ramos, Tomás Almeida, Daniel Vareta, Filipe Azevedo, Sweta Agrawal, Patrick Fernandes, André F. T. Martins
| Challenge: | Reinforcement learning (RL) is an effective and robust method for training neural machine translation systems. |
| Approach: | They propose a method that leverages fine-grained, token-level quality assessments . they use a state-of-the-art quality estimation system as their token- level reward model . |
| Outcome: | The proposed approach leverages fine-grained, token-level quality assessments along with error severity levels to improve translation quality. |
Watching the Watchers: Exposing Gender Disparities in Machine Translation Quality Estimation (2025.acl-long)
Copied to clipboard
| Challenge: | Qualitative estimation (QE) metrics have been optimized to align with human quality judgments, but whether they encode social biases has been largely overlooked. |
| Approach: | They define and investigate gender bias of QE metrics and discuss its downstream implications for machine translation (MT) when a human entity’s gender in the source is undisclosed, masculine-inflected translations score higher than feminine-infflectes translations are penalized. |
| Outcome: | The proposed measures are based on gender-based quality estimation metrics across multiple domains, datasets, and languages. |
AfriMTE and AfriCOMET: Enhancing COMET to Embrace Under-resourced African Languages (2024.naacl-long)
Copied to clipboard
Jiayi Wang, David Adelani, Sweta Agrawal, Marek Masiak, Ricardo Rei, Eleftheria Briakou, Marine Carpuat, Xuanli He, Sofia Bourhim, Andiswa Bukula, Muhidin Mohamed, Temitayo Olatoye, Tosin Adewumi, Hamam Mokayed, Christine Mwase, Wangui Kimotho, Foutse Yuehgoh, Anuoluwapo Aremu, Jessica Ojo, Shamsuddeen Muhammad, Salomey Osei, Abdul-Hakeem Omotayo, Chiamaka Chukwuneke, Perez Ogayo, Oumaima Hourrane, Salma El Anigri, Lolwethu Ndolela, Thabiso Mangwana, Shafie Mohamed, Hassan Ayinde, Oluwabusayo Awoyomi, Lama Alkhaled, Sana Al-azzawi, Naome Etori, Millicent Ochieng, Clemencia Siro, Njoroge Kiragu, Eric Muchiri, Wangari Kimotho, Toadoum Sari Sakayo, Lyse Naomi Wamba, Daud Abolade, Simbiat Ajao, Iyanuoluwa Shode, Ricky Macharm, Ruqayya Iro, Saheed Abdullahi, Stephen Moore, Bernard Opoku, Zainab Akinjobi, Abeeb Afolabi, Nnaemeka Obiefuna, Onyekachi Ogbu, Sam Ochieng’, Verrah Otiende, Chinedu Mbonu, Yao Lu, Pontus Stenetorp
| Challenge: | Recent advances in machine translation (MT) have focused on scaling multilingual machine translation models and evaluation data to hundreds of languages, including multiple under-resourced languages. |
| Approach: | They propose to use n-gram matching metrics to measure progress in multilingual machine translation to 13 typologically diverse African languages to create high-quality human evaluation data with simplified MQM guidelines. |
| Outcome: | The proposed metrics have a higher correlation with human judgments than n-gram matching metrics such as BLEU and METEOR. |
xTower: A Multilingual LLM for Explaining and Correcting Translation Errors (2024.findings-emnlp)
Copied to clipboard
Marcos Treviso, Nuno Guerreiro, Sweta Agrawal, Ricardo Rei, José Pombal, Tania Vaz, Helena Wu, Beatriz Silva, Daan Stigt, Andre Martins
| Challenge: | Neural machine translation systems produce translations with errors and anomalies . understanding these errors can help improve the translation quality and user experience . |
| Approach: | They propose an open large language model (LLM) built on top of TowerBase to provide free-text explanations for translation errors in order to guide the generation of a corrected translation. |
| Outcome: | The proposed model improves translation quality and user experience by allowing translators to provide free-text explanations for errors and anomalies. |
Can Automatic Metrics Assess High-Quality Translations? (2024.emnlp-main)
Copied to clipboard
| Challenge: | a recent human evaluation study found that translations produced by current MT systems achieve very high-quality scores when judged by humans on a direct assessment scale of 0 to 100. |
| Approach: | They stress-test the ability of current translation quality metrics to detect correct translations . they show that current metrics often over or underestimate translation quality . |
| Outcome: | The proposed method overestimates translation quality, the authors show . they show that current metrics often overestimate translation quality . |
Physician Detection of Clinical Harm in Machine Translation: Quality Estimation Aids in Reliance and Backtranslation Identifies Critical Errors (2023.emnlp-main)
Copied to clipboard
| Challenge: | a major challenge in the practical use of Machine Translation (MT) is that users lack information on translation quality to make informed decisions about how to rely on outputs. |
| Approach: | They evaluate quality estimation feedback in vivo with a human study in a medical setting. |
| Outcome: | The proposed method improves appropriate reliance on MT, but backtranslation helps detect harmful errors. |
Controlling Pre-trained Language Models for Grade-Specific Text Simplification (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to text simplification control output complexity at corpus level disregarding complexity of individual inputs and considering only one level of output complexity. |
| Approach: | They propose a method that predicts edit operations required for a specific grade level . they say this approach improves the quality of the simplified outputs over corpus-level heuristics . |
| Outcome: | The proposed method improves the readability of simplified outputs over corpus-level search-based heuristics. |
In-context Examples Selection for Machine Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | Large-scale generative models can perform a wide range of NLP tasks using in-context learning. |
| Approach: | They aim to understand the properties of good in-context examples for machine translation in both in-domain and out-of-domain settings. |
| Outcome: | The proposed model outperforms a strong kNN-MT baseline in 2 out of 4 out-of-domain datasets. |
BLESS: Benchmarking Large Language Models on Sentence Simplification (2023.emnlp-main)
Copied to clipboard
Tannon Kew, Alison Chi, Laura Vásquez-Rodríguez, Sweta Agrawal, Dennis Aumiller, Fernando Alva-Manchego, Matthew Shardlow
| Challenge: | BLESS is a performance benchmark of the most recent state-of-the-art Large Language Models (LLMs) on the task of text simplification (TS). |
| Approach: | They present a performance benchmark of the most recent state-of-the-art Large Language Models (LLMs) on the task of text simplification (TS). |
| Outcome: | The proposed benchmarks show that the most recent state-of-the-art LLMs perform better on the task of text simplification (TS). |
Modeling User Preferences with Automatic Metrics: Creating a High-Quality Preference Dataset for Machine Translation (2024.emnlp-main)
Copied to clipboard
Sweta Agrawal, José De Souza, Ricardo Rei, António Farinhas, Gonçalo Faria, Patrick Fernandes, Nuno Guerreiro, Andre Martins
| Challenge: | Existing algorithms for machine translation do not match human preferences, but they can be expensive to obtain and curate at a large scale. |
| Approach: | They propose an approach that leverages the best of both worlds by collecting sentence-level quality assessments from professional linguists on translations generated by multiple high-quality MT systems. |
| Outcome: | The proposed approach improves translation quality on WMT23 and FLORES benchmarks. |