7 Points to Tsinghua but 10 Points to ? Assessing Large Language Models in Agentic Multilingual National Bias (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models have garnered significant attention for their capabilities in multilingual natural language processing, but studies on risks associated with cross biases are limited to immediate context preferences. |
| Approach: | They investigate multilingual bias in state-of-the-art Large Language Models by analyzing their responses to decision-making tasks across multiple languages. |
| Outcome: | The proposed model can provide personalized advice across university applications, travel, and relocation scenarios. |
Similar Papers
Are Large Language Model-based Evaluators the Solution to Scaling Up Multilingual Evaluation? (2024.findings-eacl)
Copied to clipboard
Rishav Hada, Varun Gumma, Adrian Wynter, Harshita Diddee, Mohamed Ahmed, Monojit Choudhury, Kalika Bali, Sunayana Sitaram
| Challenge: | Large Language Models (LLMs) excel in various tasks, but their evaluation, especially in languages beyond the top 20, remains inadequate due to existing benchmarks and metrics limitations. |
| Approach: | They propose to use Large Language Models as evaluators to rank or score other models’ outputs by calibrating them against 20K human judgments across three text-generation tasks, five metrics, and eight languages. |
| Outcome: | The proposed evaluation methods can be used to improve multilingual evaluation by calibrating them against 20K human judgments across three text-generation tasks, five metrics, and eight languages. |
Mind the Gap: Multilingual Divide in LLM Bias Detection and Reasoning (2026.acl-srw)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly deployed in multilingual settings . but most bias evaluation remains English-centric and ignores how bias manifests within reasoning . |
| Approach: | They evaluate large language models with supervised fine-tuning and preference optimization . they find that bias varies substantially across languages, with consistent degradation in non-English settings . |
| Outcome: | The proposed model improves in English, Dutch, Spanish, and Turkish using the MBBQ benchmark. |
The Linguistic Connectivities Within Large Language Models (2025.findings-acl)
Copied to clipboard
Dan Wang, Boxi Cao, Ning Bian, Xuanang Chen, Yaojie Lu, Hongyu Lin, Jia Zheng, Le Sun, Shanshan Jiang, Bin Dong, Xianpei Han
| Challenge: | Recent studies have discovered notable disparities in their performance across different languages. |
| Approach: | They conduct a systematic investigation into the behaviors of large language models across 27 different languages on 3 different scenarios and reveals a Linguistic Map correlates with the richness of available resources and linguistic family relations. |
| Outcome: | The proposed model demonstrates that there are significant disparities in performance across languages across 27 different languages on 3 different scenarios. |
Location Not Found: Exposing Implicit Local and Global Biases in Multilingual LLMs (2026.acl-long)
Copied to clipboard
Guy Mor-Lan, Omer Goldman, Matan Eyal, Adi Mayrav Gilady, Sivan Eiger, Idan Szpektor, Avinatan Hassidim, Yossi Matias, Reut Tsarfaty
| Challenge: | Multilingual large language models have minimized the fluency gap between languages, but they are exposed to the risk of biases as knowledge and norms may propagate across languages. |
| Approach: | They propose a test set with 2,156 questions in 12 languages to quantify models' biases . they show a global bias towards answers relevant to the US-locale . |
| Outcome: | The proposed model can answer locale-ambiguous questions in 12 languages. |
Navigating the Political Compass: Evaluating Multilingual LLMs across Languages and Nationalities (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are ubiquitous in today’s technological landscape, boasting a plethora of applications, and even endangering human jobs in complex and creative fields. |
| Approach: | They evaluate the political bias of 15 multilingual LLMs using the Political Compass Test and assign a nationality to each model. |
| Outcome: | The models on the 50 most populous countries and their official languages exhibit political bias. |
Social Bias in Multilingual Language Models: A Survey (2025.emnlp-main)
Copied to clipboard
| Challenge: | Pretrained multilingual models exhibit the same social bias as models processing English texts. |
| Approach: | They examine the literature on bias evaluation and mitigation approaches in multilingual and non-English contexts and identify gaps in the field. |
| Outcome: | The proposed models perform well on multilingual language understanding benchmarks and are consistent with the current literature. |
Do Large Language Models have an English Accent? Evaluating and Improving the Naturalness of Multilingual LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Current Large Language Models (LLMs) are predominantly designed with English as the primary language, but many are still English-dominated. |
| Approach: | They propose to use automatic corpus-level metrics to assess lexical and syntactic naturalness of LLMs in a multilingual context. |
| Outcome: | The proposed method improves naturalness of LLMs in target languages without compromising performance on general-purpose benchmarks. |
Is Translation All You Need? A Study on Solving Multilingual Tasks with Large Language Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated multilingual capabilities, yet they are mostly English-centric due to the imbalanced training corpora. |
| Approach: | They extend the evaluation to real-world user queries and non-English-centric LLMs . they show that translation into English can boost LLM performance on NLP tasks . |
| Outcome: | The proposed evaluation extends to user queries and non-English-centric LLMs . it shows that translation into English can boost performance on NLP tasks, but not universally optimal . |
Safety of Large Language Models Beyond English: A Systematic Literature Review of Risks, Biases, and Safeguards (2026.eacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have a growing number of applications that generate harmful, biased, or unsafe content. |
| Approach: | They synthesize findings from recent studies that evaluate their robustness across languages . they highlight gaps in multilingual safety research and recommend future work . |
| Outcome: | The systematic review examines the multilingual safety of large language models in English . it identifies challenges such as dataset availability and evaluation biases . |
Do LLMs Align Human Values Regarding Social Biases? Judging and Explaining Social Biases with LLMs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models can lead to undesired consequences when misaligned with human values . previous studies have shown misalignment of LLMs with human value using expert-designed or agent-based emulated bias scenarios . |
| Approach: | They investigate whether large language models (LLMs) are misaligned with human values . they find no significant differences in understanding of HVSB between LLMs . |
| Outcome: | The results show that large language models do not have lower misalignment rates and attack success rates . the study also shows that smaller language models have the ability to explain HVSB . |