Papers by Beatrice Savoldi
Gender in Danger? Evaluating Speech Translation Technology on the MuST-SHE Corpus (2020.acl-main)
Copied to clipboard
| Challenge: | a growing number of studies have examined the issue of gender bias in speech translation . a gender bias is a systemic problem that reproduces gender stereotypes discriminating women. |
| Approach: | They present the first thorough investigation of gender bias in speech translation . they compare audio technologies for English-Italian/French translations . |
| Outcome: | The proposed method compares different technologies on two languages, English and French. |
Hi Guys or Hi Folks? Benchmarking Gender-Neutral Machine Translation with the GeNTE Corpus (2023.emnlp-main)
Copied to clipboard
| Challenge: | Societal gender asymmetries and inequalities are perpetuated through language . MT often defaults to masculine representations by making undue binary gender assumptions . |
| Approach: | They propose a benchmark and automated evaluation methods to assess gender-neutral translation from English to Italian. |
| Outcome: | The proposed method is based on a survey on gender-neutral translation. |
What the Harm? Quantifying the Tangible Impact of Gender Bias in Machine Translation with a Human-centered Study (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing bias measurements do not reflect the gender disparities found in machine translation. |
| Approach: | They conduct a human-centered study to examine if and to what extent bias in machine translation brings harms with tangible costs, such as quality of service gaps between women and men. |
| Outcome: | The findings advocate for human-centered approaches that can inform the societal impact of bias. |
How to Split: the Effect of Word Segmentation on Gender Bias in Speech Translation (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for subword splitting penalize the representation of feminine linguistic markings. |
| Approach: | They propose a method that preserves subword splitting while leveraging character-based segmentation to properly translate gender. |
| Outcome: | The proposed approach preserves BPE overall translation quality while leveraging the higher ability of character-based segmentation to properly translate gender. |
Twists, Humps, and Pebbles: Multilingual Speech Recognition Models Exhibit Gender Performance Gaps (2024.emnlp-main)
Copied to clipboard
| Challenge: | a new class of multitasks, multilingual neural networks, has recently pushed the boundaries of speech-related tasks. |
| Approach: | They evaluate performance of two widely used multilingual automatic speech recognition models . they find clear gender disparities, with the advantaged group varying across languages . |
| Outcome: | The proposed models are compared on 19 languages from eight language families and two speaking conditions. |
Gender Bias in Machine Translation (2021.tacl-1)
Copied to clipboard
| Challenge: | Interest in understanding, assessing, and mitigating gender bias in machine translation (MT) still lacks cohesion. |
| Approach: | They propose to review current conceptualizations of gender bias in machine translation (MT) they summarize previous studies and propose ways to mitigate bias. |
| Outcome: | This paper summarizes the current conceptualizations and proposes strategies to mitigate biases in machine translation (MT) . |
Under the Morphosyntactic Lens: A Multifaceted Evaluation of Gender Bias in Speech Translation (2022.acl-long)
Copied to clipboard
| Challenge: | grammatical gender languages are characterized by morphosyntactic chains of gender agreement marked on a variety of lexical items and parts-of-speech (POS). |
| Approach: | They propose to enrich the natural, gender-sensitive MuST-SHE corpus with two new linguistic annotation layers to explore gender bias. |
| Outcome: | The proposed models shed light on gender bias and its detection at several levels of granularity. |
Translation in the Hands of Many: Centering Lay Users in Machine Translation Interactions (2025.emnlp-main)
Copied to clipboard
| Challenge: | Multilingual demands and accessibility have made MT a global tool . however, the understanding of MT consumed by such a diverse group of users remains limited. |
| Approach: | They first trace the evolution of MT user profiles, focusing on non-experts and how their engagement with technology may shift with the rise of LLMs. |
| Outcome: | The proposed approach will help to align MT with user needs and improve the quality of the language. |
Mind the Inclusivity Gap: Multilingual Gender-Neutral Translation Evaluation with mGeNTE (2025.emnlp-main)
Copied to clipboard
Beatrice Savoldi, Giuseppe Attanasio, Eleonora Cupin, Eleni Gkovedarou, Janiça Hackenbuchner, Anne Lauscher, Matteo Negri, Andrea Piergentili, Manjinder Thind, Luisa Bentivogli
| Challenge: | Genderneutral translation (GNT) is a linguistic strategy towards fairer communication across languages. |
| Approach: | They propose to use a multilingual evaluation resource to evaluate inclusive translation with state-of-the-art instruction-following language models (LMs) |
| Outcome: | The proposed model can recognize when neutrality is appropriate, but cannot consistently produce neutral translations, limiting their usability. |
A Prompt Response to the Demand for Automatic Gender-Neutral Translation (2024.eacl-short)
Copied to clipboard
| Challenge: | Advancements in machine translation (MT) are hindered by the lack of dedicated parallel data, which are necessary to adapt MT systems to satisfy neutral constraints. |
| Approach: | They propose to use GPT-4 to generate GNTs that avoid bias and undue binary assumptions by comparing MT with the popular GPT-3 model. |
| Outcome: | The proposed model outperforms the existing model and provides valuable insights into the potential and challenges associated with prompting for neutrality. |
SHADES: Towards a Multilingual Assessment of Stereotypes in Large Language Models (2025.naacl-long)
Copied to clipboard
Margaret Mitchell, Giuseppe Attanasio, Ioana Baldini, Miruna Clinciu, Jordan Clive, Pieter Delobelle, Manan Dey, Sil Hamilton, Timm Dill, Jad Doughman, Ritam Dutt, Avijit Ghosh, Jessica Zosa Forde, Carolin Holtermann, Lucie-Aimée Kaffee, Tanmay Laud, Anne Lauscher, Roberto L Lopez-Davila, Maraim Masoud, Nikita Nangia, Anaelia Ovalle, Giada Pistilli, Dragomir Radev, Beatrice Savoldi, Vipul Raheja, Jeremy Qin, Esther Ploeger, Arjun Subramonian, Kaustubh Dhole, Kaiser Sun, Amirbek Djanibekov, Jonibek Mansurov, Kayo Yin, Emilio Villa Cueva, Sagnik Mukherjee, Jerry Huang, Xudong Shen, Jay Gala, Hamdan Al-Ali, null Tair Djanibekov, Nurdaulet Mukhituly, Shangrui Nie, Shanya Sharma, Karolina Stanczak, Eliza Szczechla, Tiago Timponi Torrent, Deepak Tunuguntla, Marcelo Viridiano, Oskar Van Der Wal, Adina Yakefu, Aurélie Névéol, Mike Zhang, Sydney Zink, Zeerak Talat
| Challenge: | Large Language Models reproduce and exacerbate social biases present in training data, and resources to quantify this issue are limited. |
| Approach: | They propose a multilingual parallel dataset to examine culturally-specific stereotypes that may be learned by LLMs. |
| Outcome: | The proposed dataset includes stereotypes from 20 regions around the world and 16 languages, spanning multiple identity categories subject to discrimination worldwide. |
Breeding Gender-aware Direct Speech Translation Systems (2020.coling-main)
Copied to clipboard
| Challenge: | In automatic speech translation, traditional cascade approaches involving separate transcription and translation steps are giving ground to more robust direct solutions. |
| Approach: | They compare different approaches to inform direct ST models about the speaker’s gender and test their ability to handle gender translation from English into Italian and French. |
| Outcome: | The proposed models outperform strong but gender-unaware direct ST models in the translation of English into Italian and French. |