Safety of Large Language Models Beyond English: A Systematic Literature Review of Risks, Biases, and Safeguards (2026.eacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have a growing number of applications that generate harmful, biased, or unsafe content. |
| Approach: | They synthesize findings from recent studies that evaluate their robustness across languages . they highlight gaps in multilingual safety research and recommend future work . |
| Outcome: | The systematic review examines the multilingual safety of large language models in English . it identifies challenges such as dataset availability and evaluation biases . |
Similar Papers
A Chinese Dataset for Evaluating the Safeguards in Large Language Models (2024.findings-acl)
Copied to clipboard
Yuxia Wang, Zenan Zhai, Haonan Li, Xudong Han, Shom Lin, Zhenxuan Zhang, Angela Zhao, Preslav Nakov, Timothy Baldwin
| Challenge: | a recent study has shown that large language models can produce harmful responses, exposing users to unexpected risks. |
| Approach: | They propose a dataset for the safety evaluation of Chinese LLMs in Mandarin Chinese . they extend the dataset to better identify false negative and false positive examples . |
| Outcome: | The proposed dataset is for the safety evaluation of Chinese LLMs, and is based on a Chinese dataset. |
The Language Barrier: Dissecting Safety Challenges of LLMs in Multilingual Contexts (2024.findings-acl)
Copied to clipboard
Lingfeng Shen, Weiting Tan, Sihao Chen, Yunmo Chen, Jingyu Zhang, Haoran Xu, Boyuan Zheng, Philipp Koehn, Daniel Khashabi
| Challenge: | Recent studies show that malicious prompt instructions could solicit objectionable content from LLMs. |
| Approach: | They compare how state-of-the-art LLMs respond to malicious prompts in different languages . they find that LLM's generate unsafe responses more often when a prompt is written in a lower-resource language . |
| Outcome: | The proposed model can generate unsafe responses more often when a malicious prompt is written in a lower-resource language, and less irrelevant responses when written in lower-source languages. |
A Systematic Survey and Critical Review on Evaluating Large Language Models: Challenges, Limitations, and Recommendations (2024.emnlp-main)
Copied to clipboard
Md Tahmid Rahman Laskar, Sawsan Alqahtani, M Saiful Bari, Mizanur Rahman, Mohammad Abdullah Matin Khan, Haidar Khan, Israt Jahan, Amran Bhuiyan, Chee Wei Tan, Md Rizwan Parvez, Enamul Hoque, Shafiq Joty, Jimmy Huang
| Challenge: | Large Language Models (LLMs) have gained significant attention due to their capabilities in performing diverse tasks across domains. |
| Approach: | They review the primary challenges and limitations causing inconsistencies in evaluations . early models could generate coherent text but limited to simple tasks . |
| Outcome: | The proposed evaluations are reproducible, reliable, and robust. |
The State of Multilingual LLM Safety Research: From Measuring The Language Gap To Mitigating It (2025.emnlp-main)
Copied to clipboard
| Challenge: | a systematic review of 300 publications reveals a language gap in LLM safety research . even high-resource non-English languages receive little attention, authors note . |
| Approach: | They propose to focus on safety evaluation, training data generation, and crosslingual safety generalization based on their findings. |
| Outcome: | The authors suggest that the field can develop more robust, inclusive safety practices for diverse global populations. |
Everything you need to know about Multilingual LLMs: Towards fair, performant and reliable models for languages of the world (2023.acl-tutorials)
Copied to clipboard
| Challenge: | Responsible AI issues such as fairness, bias and toxicity will be discussed in this tutorial . |
| Approach: | This tutorial will describe various aspects of scaling up language technologies to many of the world’s languages by describing the latest research in Massively Multilingual Language Models (MMLMs). |
| Outcome: | This tutorial will cover various aspects of scaling up language technologies to many of the world's languages by describing the latest research in multilingual models. |
Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safety (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing surveys focus on interpretation or safety, but safety and understanding are core motivations for interpretation research. |
| Approach: | They propose a framework that connects interpretation methods, enhancements they inform, and tools that operationalize them. |
| Outcome: | The proposed framework summarizes nearly 70 studies at their intersections and concludes with open challenges and future directions. |
Awes, Laws, and Flaws From Today’s LLM Research (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are a powerful technology that can follow instructions and output coherent, persuasive text. |
| Approach: | They examine the scientific methodology behind large language model (LLM) research and cross-validate it with arguments at the centre of controversy. |
| Outcome: | The authors cross-validate 2,000 research works released between 2020 and 2024 based on criteria typical of what is considered good research and find that conference checklists are effective at curtailing some of these issues, but balancing velocity and rigour in research cannot solely rely on these. |
Is Safety Standard Same for Everyone? User-Specific Safety Evaluation of Large Language Models (2025.findings-emnlp)
Copied to clipboard
Yeonjun In, Wonjoong Kim, Kanghoon Yoon, Sungchul Kim, Mehrab Tanjim, Sangwu Park, Kibum Kim, Chanyoung Park
| Challenge: | Extensive benchmarks evaluate LLM safety relying heavily on general standards . no benchmark datasets exist to evaluate the user-specific safety of LLMs . |
| Approach: | a new benchmark is designed to assess user-specific aspect of LLM safety . authors propose a simple remedy based on chain-of-thought to improve user-specified safety. |
| Outcome: | a new benchmark assesses the user-specific aspect of LLM safety . the proposed solution improves user-specified safety by chain-of-thought . |
Vulnerabilities of Large Language Models to Adversarial Attacks (2024.acl-tutorials)
Copied to clipboard
| Challenge: | This tutorial focuses on the vulnerabilities of Large Language Models to adversarial attacks . the tutorial lays the foundation by explaining safety-aligned models and concepts in cybersecurity . |
| Approach: | This tutorial lays the foundation by explaining safety-aligned LLMs and concepts in cybersecurity. |
| Outcome: | The tutorial lays the foundation by explaining safety-aligned models and concepts in cybersecurity. |
A Survey of Toxicity Mitigation Strategies for Multilingual Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models can reproduce and amplify toxic content, including hate speech, harassment, and bias. |
| Approach: | They propose a comprehensive survey of the many detoxification methods tailored to multilingual LLMs. |
| Outcome: | The proposed methods are based on data filtering, style transfer, expert-based logit steering, retrieval augmentation, and human feedback. |