The Rise of Darkness: Safety-Utility Trade-Offs in Role-Playing Dialogue Agents (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) demonstrate their utility in character simulations, but they pose a risk of generating unsafe content. |
| Approach: | They propose a method which dynamically adjusts safety-utility preferences based on the degree of risk coupling and guides the model to generate responses biased toward utility or safety. |
| Outcome: | The proposed method improves safety metrics while maintaining utility. |
Similar Papers
Beware of Your Po! Measuring and Mitigating AI Safety Risks in Role-Play Fine-Tuning of LLMs (2025.acl-long)
Copied to clipboard
Weixiang Zhao, Yulin Hu, Yang Deng, Jiahe Guo, Xingyu Sui, Xinyang Han, An Zhang, Yanyan Zhao, Bing Qin, Tat-Seng Chua, Ting Liu
| Challenge: | Existing role-play fine-tuning techniques improve role adaptability but may degrade safety performance, especially for villainous characters. |
| Approach: | They propose safety-aware Role-Play Fine-Tuning (SaRFT) to balance role-playing capabilities and safety. |
| Outcome: | The proposed method outperforms state-of-the-art baselines under both LoRA and full-parameter fine-tuning settings. |
Exploring Safety-Utility Trade-Offs in Personalized Language Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Prior studies have shown that large language models can exhibit bias against specific demographic groups and engage in the generation of stereotypical responses. |
| Approach: | They propose a framework to evaluate LLM performance along two axes: safety and utility. |
| Outcome: | The proposed framework evaluates the performance of LLMs along two axes: safety and utility. |
SafeSearch: Do Not Trade Safety for Utility in LLM Search Agents (2026.findings-eacl)
Copied to clipboard
| Challenge: | Large language model (LLM) based search agents are more likely to produce harmful outputs than base models. |
| Approach: | They propose a query-level shaping term that rewards safe queries and penalizes unsafe ones. |
| Outcome: | The proposed approach reduces harmfulness by over 70% across three red-teaming datasets while producing safe, helpful responses. |
Too Good to be Bad: On the Failure of LLMs to Role-Play Villains (2026.findings-acl)
Copied to clipboard
Zihao Yi, Qingxuan Jiang, Ruotian Ma, Xingyu Chen, Qu Yang, Mengru Wang, Fanghua Ye, Ying Shen, Zhaopeng Tu, Xiaolong Li, Liefeng Bo
| Challenge: | Large Language Models (LLMs) are increasingly tasked with creative generation, but their ability to portray non-prosocial, antagonistic personas remains largely unexamined. |
| Approach: | They propose a moral alignment benchmark to test the safety of large language models . they find that models struggle with traits directly antithetical to safety principles . |
| Outcome: | The proposed model fails to accurately portray morally ambiguous or villainous characters . the model fails most with traits directly antithetical to safety principles . |
Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safety (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing surveys focus on interpretation or safety, but safety and understanding are core motivations for interpretation research. |
| Approach: | They propose a framework that connects interpretation methods, enhancements they inform, and tools that operationalize them. |
| Outcome: | The proposed framework summarizes nearly 70 studies at their intersections and concludes with open challenges and future directions. |
MTMCS-Bench: Evaluating Contextual Safety of Multimodal Large Language Models in Multi-Turn Dialogues (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing contextual safety benchmarks are mostly single-turn and miss how malicious intent can emerge gradually or how the same scene can support both benign and exploitative goals. |
| Approach: | They propose a benchmark that evaluates contextual safety in multimodal large language models . they observe persistent trade-offs between contextual safety and utility . |
| Outcome: | The proposed model combines multi-turn and multi-switch scenarios to evaluate safety in multimodal large language models. |
SAFER: Advancing Safety Alignment via Efficient Ex-Ante Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing alignment methods struggle to cover diverse safety scenarios and remain vulnerable to adversarial attacks. |
| Approach: | They propose a framework for 'S**afety' alignment via e**F**ficient' E**x-Ante-R**easoning that instantiates structured Ex-Ance reasoning and embeds predefined safety rules. |
| Outcome: | The proposed framework enhances safety performance while maintaining usefulness and efficiency. |
Beyond Reactive Safety: Risk-Aware LLM Alignment via Long-Horizon Simulation (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing alignment methods focus on reactive feedback, where immediate human perception is leveraged to judge sampled model responses as preference data for post-training. |
| Approach: | They propose a proof-of-concept framework that projects how model-generated advice could propagate through societal systems on a macroscopic scale over time, enabling more robust alignment. |
| Outcome: | The proposed framework achieves 20% improvement on existing safety benchmarks and an average win rate exceeding 70% against strong baselines. |
SafeMCP: Proactive Power Regulation for LLM Agent Defense via Environment-Grounded Look-Ahead Reasoning (2026.acl-long)
Copied to clipboard
| Challenge: | Large Language Model (LLM) agents are expanding their action spaces to operate in complex environments. |
| Approach: | They propose a server-side defense plugin that constrains tool acquisition via predictive reasoning regarding future safety risks. |
| Outcome: | Experiments on PowerSeeking Bench, ToolEmu, and AgentHarm show that SafeMCP achieves a safe equilibrium, effectively mitigating risks while preserving agent utility. |
Preventing Safety Drift in Large Language Models via Coupled Weight and Activation Constraints (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing defenses constrain either weights or activations in isolation, without considering their coupled effects on safety. |
| Approach: | They propose a weight-activation constraint that enforces a precomputed safety subspace on weight updates and applies regularization to safety-critical features identified by sparse autoencoders. |
| Outcome: | The proposed model outperforms baselines even under high harmful data ratios. |