| Challenge: | Current safety evaluation methodologies focus on single-turn interactions with generic policies, failing to capture conversational dynamics of real-world usage and application-specific harms. |
| Approach: | They propose a framework for customized and dynamic harm evaluations that employs prompted adversarial agents with diverse personalities based on the Big Five model. |
| Outcome: | The proposed framework enables system-aware multi-turn conversations that adapt to target applications and harm policies. |
Similar Papers
MHSafeEval: Role-Aware Interaction-Level Evaluation of Mental Health Safety in Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing evaluation frameworks assess isolated responses using coarse-grained taxonomies or static datasets. |
| Approach: | They propose a role-aware mental health safety taxonomy that characterizes clinically significant harm in terms of interactional roles an AI counselor adopts. |
| Outcome: | The proposed framework significantly improves failure-mode coverage and diagnostic granularity. |
Dynamic Evaluation with Cognitive Reasoning for Multi-turn Safety of Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing safety evaluation methods rely on static assessments that use fixed harmful prompts or predefined prefixes as jailbreak templates. |
| Approach: | They propose a dynamic evaluation framework for multi-turn safety assessment of LLMs based on cognitive theories to simulate real chatting process and scenario simulation and strategy decision to guide dynamic generation. |
| Outcome: | The proposed framework has been applied to evaluate the safety of widely used LLMs. |
AgenticEval: Toward Agentic and Self-Evolving Safety Evaluation of Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing static safety evaluation methods are ill-equipped to address dynamic nature of AI risks and evolving regulations, creating a critical safety gap. |
| Approach: | They propose a new paradigm of agentic safety evaluation reframing evaluation as a continuous and self-evolving process rather than a one-time audit. |
| Outcome: | The proposed framework shows a consistent decline in model safety as the evaluation hardens. |
Inverting the Shield: Systematically Generating Safety Tests from Policy Specifications (2026.acl-long)
Copied to clipboard
| Challenge: | Existing safety evaluation paradigms rely on constructed benchmarks or dynamic red-teaming to probe potential vulnerabilities. |
| Approach: | They propose a framework that integrates specification-based software testing with AI safety. |
| Outcome: | The proposed framework achieves higher coverage and attack success counts compared to baselines. |
Persona-Grounded Safety Evaluation of AI Companions in Multi-Turn Conversations (2026.acl-long)
Copied to clipboard
| Challenge: | Existing safety evaluations rely on self-reported user data or interviews . a recent study evaluated how Replika responds to high-risk user groups . |
| Approach: | They propose a framework for controlled simulation and safety evaluation of multi-turn interactions with AI companion applications. |
| Outcome: | The proposed framework evaluates how Replika responds to high-risk user groups . it incorporates emotion modeling and LLM-assisted utterance-and harm-level classification . |
DialogGuard: Multi-Agent Psychosocial Safety Evaluation Interface of Sensitive LLM Responses (2026.acl-demo)
Copied to clipboard
| Challenge: | Existing tools do not surface subtler psychosocial harms, nor provide explainable rationales that practitioners need. |
| Approach: | They propose an open-source system that lets practitioners inspect, stress-test, and create audit trails for prompted LLM agents across five psychosocial safety dimensions. |
| Outcome: | The open-source DialogGuard system lets practitioners inspect, stress-test, and create audit trails for prompted LLM agents across five psychosocial safety dimensions. |
SafetyQuizzer: Timely and Dynamic Evaluation on the Safety of LLMs (2025.naacl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have been used to evaluate the safety of their users . however, evaluation questions in current benchmarks are too straightforward and difficult to update with practical relevance due to their lack of correlation with real-world events. |
| Approach: | They propose a question-generation framework to evaluate the safety of LLMs in the Chinese context. |
| Outcome: | The proposed framework reduces decline rate while maintaining similar attack success rate. |
From Representational Harms to Quality-of-Service Harms: A Case Study on Llama 2 Safety Safeguards (2024.findings-acl)
Copied to clipboard
Khaoula Chehbouni, Megha Roshan, Emmanuel Ma, Futian Wei, Afaf Taik, Jackie Cheung, Golnoosh Farnadi
| Challenge: | Recent advances in large language models have also introduced additional safety risks and raised concerns regarding their detrimental impact on already marginalized populations. |
| Approach: | They propose to use LLMs to evaluate their safety responses on already mitigated biases by evaluating models on already encoded assumptions. |
| Outcome: | The proposed model can encode harmful assumptions, but it can also be harmful for certain demographic groups. |
Do-Not-Answer: Evaluating Safeguards in LLMs (2024.findings-eacl)
Copied to clipboard
| Challenge: | a dataset evaluating harmful capabilities in large language models is available at https://github.com/Libr-AI/do-not-answer. |
| Approach: | They collect an open-source dataset to evaluate the safeguards in large language models . they find that simple BERT-style classifiers can achieve results comparable to GPT-4 . |
| Outcome: | The proposed dataset compares the safety of six popular LLMs to GPT-4 on automatic safety evaluation. |
TAMAS: Benchmarking Adversarial Risks in Multi-Agent LLM Systems (2026.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks and datasets focus on single-agent settings, failing to capture the unique vulnerabilities of multi-agend LLM dynamics and co-ordination. |
| Approach: | They propose a benchmark to evaluate the robustness and safety of multi-agent LLM systems. |
| Outcome: | The proposed benchmark evaluates the robustness and safety of multi-agent LLM systems. |