Chenyu Shi, Xiao Wang, Qiming Ge, Songyang Gao, Xianjun Yang, Tao Gui, Qi Zhang, Xuanjing Huang, Xun Zhao, Dahua Lin
| Challenge: | Recent studies have highlighted a tendency among large language models to refuse to answer benign queries. |
| Approach: | They propose a model-agnostic approach to reduce excessive attention to harmful words like ‘kill’ and a method to decode the next-token predictions by contrastive decoding. |
| Outcome: | The proposed approach reduces the refusal rate by 20% while having little impact on safety. |
Similar Papers
Please refuse to answer me! Mitigating Over-Refusal in Large Language Models via Adaptive Contrastive Decoding (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for mitigating over-refusal can't maintain low refusal ratio for harmless queries while keeping high for malicious queries. |
| Approach: | They propose a model-agnostic approach to mitigate over-refusal in large language models . they propose an adaptive contrastive decoding strategy that incorporates or removes the refusal token distribution . |
| Outcome: | The proposed approach reduces the refusal ratio for over-refusal queries by 10.35% while increasing the refusal rate for malicious queries by 0.13%. |
Understanding and Mitigating Overrefusal in LLMs from an Unveiling Perspective of Safety Decision Boundary (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) often refuse to answer legitimate queries, causing models to treat many reasonable prompts as potentially risky. |
| Approach: | They propose a framework that automatically generates and selects overrefusal prompts near the safety boundary. |
| Outcome: | The proposed framework identifies and curates boundary-aligned prompts, enabling more effective and targeted mitigation of overrefusal. |
XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) are now being used by millions of people across the world. |
| Approach: | They propose a test suite called XSTest to identify such eXaggerated Safety behaviours in a systematic way. |
| Outcome: | The proposed test suite identifies eXaggerated Safety behaviours in a systematic way. |
Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safety (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing surveys focus on interpretation or safety, but safety and understanding are core motivations for interpretation research. |
| Approach: | They propose a framework that connects interpretation methods, enhancements they inform, and tools that operationalize them. |
| Outcome: | The proposed framework summarizes nearly 70 studies at their intersections and concludes with open challenges and future directions. |
ROSE Doesn’t Do That: Boosting the Safety of Instruction-Tuned Large Language Models with Reverse Prompt Contrastive Decoding (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for aligning LLMs output with expected safety require substantial training efforts and expensive computational resources. |
| Approach: | They propose a method to directly boost the safety of existing instruction-tuned large language models without additional training. |
| Outcome: | The proposed method improves safety of instruction-tuned large language models without training and requires expensive computational resources. |
Exploring and Mitigating Shortcut Learning for Generative Large Language Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent large language models (LLMs) have incredible instruction-following capabilities while maintaining strong task completion ability. |
| Approach: | They propose a framework to encourage LLMs to Forget Spurious correlations and Learn from In-context information. |
| Outcome: | The proposed framework can mitigate shortcut learning by forging spurious correlations and learning from in-context information. |
SLM as Guardian: Pioneering AI Safety with Small Language Model (2024.emnlp-industry)
Copied to clipboard
Ohjoon Kwon, Donghyeon Jeon, Nayoung Choi, Gyu-Hwung Cho, Hwiyeol Jo, Changbong Kim, Hyunwoo Lee, Inho Kang, Sun Kim, Taiwoo Park
| Challenge: | Prior safety research on large language models focused on aligning them to safety requirements, but internalizing such safeguard features into larger models brought challenges of higher training cost and unintended degradation of helpfulness. |
| Approach: | They propose a multi-task learning mechanism that integrates harmful query detection and safeguard response into a single model. |
| Outcome: | The proposed approach outperforms the publicly available LLMs in harmful query detection and safeguard response generation. |
Seeing No Evil: Blinding Large Vision-Language Models to Safety Instructions via Adversarial Attention Hijacking (2026.acl-long)
Copied to clipboard
| Challenge: | Existing attacks optimize image perturbations to maximize harmful output likelihood, but suffer from slow convergence due to gradient conflict between adversarial objectives and the model’s safety-retrieval mechanism. |
| Approach: | They propose a push-pull approach which suppresses attention to system-prompt tokens and anchors generation on adversarial image features to avoid collisions. |
| Outcome: | The proposed approach reduces gradient conflict by 45% and achieves 94.4% attack success rate on Qwen-VL (vs. 68.8% baseline) with 40% fewer iterations. |
E-Bench: Towards Evaluating the Ease-of-Use of Large Language Models (2025.coling-main)
Copied to clipboard
| Challenge: | E-Bench is a framework for easy-to-use research on large language models. |
| Approach: | They propose to evaluate the ease-of-use of large language models and construct an E-Bench . they simulate human use from synonymous and typographical perturbations . |
| Outcome: | The proposed model is able to resist synonymous expressions and typos and improves performance. |
Mitigating Societal Harms in Large Language Models (2023.emnlp-tutorial)
Copied to clipboard
| Challenge: | Recent studies have highlighted societal harms that can be caused by language generation models deployed in the wild. |
| Approach: | They propose to use a typology of technical approaches to mitigating harms of language generation models to provide an overview of potential social issues in language generation including toxicity, social biases, misinformation, factual inconsistency, and privacy violations. |
| Outcome: | The proposed typology addresses toxicity, biases, misinformation, factual inconsistency, and privacy violations in language generation models. |