A Multilingual, Culture-First Approach to Addressing Misgendering in LLM Applications (2025.emnlp-main)
Copied to clipboard
| Challenge: | Misgendering is the act of referring to someone by using words that do not match their chosen identity. |
| Approach: | They propose to use a participatory-design approach to assess and mitigate misgendering across 42 languages and dialects using a human-in-the-loop approach. |
| Outcome: | The proposed guardrails reduce misgendering rates across all languages and dialects without loss of quality and without loss in quality. |
Similar Papers
MisgenderMender: A Community-Informed Approach to Interventions for Misgendering (2024.naacl-long)
Copied to clipboard
| Challenge: | Misgendering is the act of incorrectly addressing someone’s gender and is pervasive in everyday use platforms and technologies. |
| Approach: | They propose a task and evaluation dataset to assess the effectiveness of automated misgendering interventions for text-based misgending in the US. |
| Outcome: | The proposed dataset includes 3790 instances of social media content and LLM-generations about non-cisgender public figures, annotated for the presence of misgendering, with additional annotations for correcting misgending in LLM generated text. |
MISGENDERED: Limits of Large Language Models in Understanding Pronouns (2023.acl-long)
Copied to clipboard
| Challenge: | excluding non-binary gender identities can perpetuate harm against non-bisexual individuals through exclusion and marginalization. |
| Approach: | They propose a framework for evaluating large language models’ ability to correctly use preferred pronouns. |
| Outcome: | The proposed framework evaluates language models' ability to correctly use preferred pronouns in English. |
On Evaluating and Mitigating Gender Biases in Multilingual Settings (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks and resources for evaluating gender biases in multilingual settings are limited. |
| Approach: | They propose to extend DisCo to different Indian languages using human annotations to evaluate gender biases in multilingual models. |
| Outcome: | The proposed benchmarks and mitigation techniques are extended beyond English to evaluate gender biases in multilingual models. |
MrGuard: A Multilingual Reasoning Guardrail for Universal LLM Safety (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are susceptible to adversarial attacks such as jailbreaking, which can elicit harmful or unsafe behaviors. |
| Approach: | They propose a multilingual guardrail with reasoning for prompt classification that integrates culturally and linguistically nuanced variants and supervised fine-tuning. |
| Outcome: | The proposed guardrail outperforms baselines across in-domain and out-of-domain languages by more than 15%. |
Social Bias in Multilingual Language Models: A Survey (2025.emnlp-main)
Copied to clipboard
| Challenge: | Pretrained multilingual models exhibit the same social bias as models processing English texts. |
| Approach: | They examine the literature on bias evaluation and mitigation approaches in multilingual and non-English contexts and identify gaps in the field. |
| Outcome: | The proposed models perform well on multilingual language understanding benchmarks and are consistent with the current literature. |
Scalable and Culturally Specific Stereotype Dataset Construction via Human-LLM Collaboration (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches for detecting and mitigating embedded stereotypes rely on carefully annotated datasets like StereoSet and CrowS-Pairs, which are only in English and reflect stereotypes from a few English-speaking countries. Existing datasets, especially translation-based ones, often overlook such cultural distinctions. |
| Approach: | They propose a cost-efficient human-LLM collaborative annotation framework to construct a Spanish-language stereotype dataset spanning multiple Spanish-speaking countries. |
| Outcome: | The proposed framework can identify nuanced, region-specific biases across Spanish-supporting LLMs and is adaptable to other languages and regions. |
Navigating the Political Compass: Evaluating Multilingual LLMs across Languages and Nationalities (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are ubiquitous in today’s technological landscape, boasting a plethora of applications, and even endangering human jobs in complex and creative fields. |
| Approach: | They evaluate the political bias of 15 multilingual LLMs using the Political Compass Test and assign a nationality to each model. |
| Outcome: | The models on the 50 most populous countries and their official languages exhibit political bias. |
EuroGEST: Investigating gender stereotypes in multilingual language models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models encode social biases, but most benchmarks for gender bias remain English-centric. |
| Approach: | They propose a dataset to measure gender-stereotypical reasoning in large language models across English and 29 European languages. |
| Outcome: | The proposed method is highly accurate across languages and strong in translations and gender labels. |
Gender Identity in Pretrained Language Models: An Inclusive Approach to Data Creation and Probing (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Pretrained language models encode binary gender information of text authors, raising the risk of skewed representations and downstream harms. |
| Approach: | They use a corpus of YouTube transcripts from transgender, cisgender and non-binary speakers to examine whether pretrained language models encode binary gender information. |
| Outcome: | The proposed model encodes gender information for all gender identities but to different extents. |
Your Stereotypical Mileage May Vary: Practical Challenges of Evaluating Biases in Multiple Languages and Cultural Contexts (2024.lrec-main)
Copied to clipboard
Karen Fort, Laura Alonso Alemany, Luciana Benotti, Julien Bezançon, Claudia Borg, Marthese Borg, Yongjian Chen, Fanny Ducel, Yoann Dupont, Guido Ivetta, Zhijian Li, Margot Mieskes, Marco Naguib, Yuyan Qian, Matteo Radaelli, Wolfgang S. Schmeisser-Nieto, Emma Raimundo Schulz, Thiziri Saci, Sarah Saidi, Javier Torroba Marchante, Shilin Xie, Sergio E. Zanotto, Aurélie Névéol
| Challenge: | Recent studies have identified a gap in the availability of tools and resources to study bias in languages other than English and social contexts outside the north of America. |
| Approach: | They use stereotypes to build a corpus of sentence pairs that cover biases in seven cultural contexts. |
| Outcome: | The proposed resource covers a wide range of languages and cultural settings . it favors sentences that express stereotypes in most bias categories . |