Han Zhu, Juntao Dai, Jiaming Ji, Haoran Li, Chengkun Cai, Pengcheng Wen, Chi-Min Chan, Boyuan Chen, Yaodong Yang, Sirui Han, Yike Guo
| Challenge: | Multi-turn dialogues pose a greater risk than single prompts, but existing safety benchmarks do not account for this situation. |
| Approach: | They propose a benchmark that features dialogues of varying lengths generated from harmful queries accompanied by images. |
| Outcome: | The proposed model reduces multi-turn Attack Success Rate (ASR) compared to existing guard models. |
Similar Papers
MTMCS-Bench: Evaluating Contextual Safety of Multimodal Large Language Models in Multi-Turn Dialogues (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing contextual safety benchmarks are mostly single-turn and miss how malicious intent can emerge gradually or how the same scene can support both benign and exploitative goals. |
| Approach: | They propose a benchmark that evaluates contextual safety in multimodal large language models . they observe persistent trade-offs between contextual safety and utility . |
| Outcome: | The proposed model combines multi-turn and multi-switch scenarios to evaluate safety in multimodal large language models. |
SafeSteer: A Decoding-level Defense Mechanism for Multimodal Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing defense methods rely on fine-tuning or inefficient post-hoc interventions, limiting their ability to address novel attacks. |
| Approach: | They propose a decoding-level defense mechanism that employs a lightweight discriminator to iteratively steer the decoding process toward safety. |
| Outcome: | The proposed method improves safety performance by up to 33.40% without fine-tuning on multiple MLLMs. |
Jailbreaking Multimodal Large Language Models using Multi-Clip Video (2026.acl-long)
Copied to clipboard
| Challenge: | Existing studies show that video inputs can bypass safety alignment, yet it remains unclear which properties of video input induce this vulnerability. |
| Approach: | They propose a simple image-based defense that mitigates the vulnerability of MLLMs by analyzing video inputs. |
| Outcome: | The proposed defense leverages the relative robustness of the image modality. |
When Helpers Become Hazards: A Benchmark for Analyzing Multimodal LLM-Powered Safety in Daily Life (2026.findings-acl)
Copied to clipboard
Xinyue Lou, Xu Jinan, Jingyi Yin, Xiaolong Wang, Zhaolu Kang, null Liaoyouwei, Yixuan Wang, Xiangyu Shi, Fengran Mo, SU Yao, Kaiyu Huang
| Challenge: | Safety impact of Multimodal Large Language Models (MLLMs) on human behavior is evaluated in this study. |
| Approach: | They propose a safety-warning-based evaluation framework that encourages models to provide clear and informative safety warnings, rather than generic refusals. |
| Outcome: | The proposed safety-warning-based evaluation framework encourages models to provide clear and informative safety warnings, rather than generic refusals. |
The Side Effects of Being Smart: Safety Risks in MLLMs’ Multi-Image Reasoning (2026.acl-long)
Copied to clipboard
Renmiao Chen, Yida Lu, Shiyao Cui, Xuan Ouyang, Victor Shea-Jay Huang, Shumin Zhang, Chengwei Pan, Han Qiu, Minlie Huang
| Challenge: | Recent advances in multimodal reasoning may pose new safety risks . evaluators neglect reasoningbased safety, where harm emerges only through MLLMs . |
| Approach: | They introduce a benchmark for multi-image reasoning safety that includes 2,676 instances . they find that models with more advanced multi- image reasoning are more vulnerable . |
| Outcome: | The proposed benchmark consists of 2,676 instances covering 9 multi-image relations . the results show that models with more advanced multi- image reasoning are more vulnerable . |
USB: A COMPREHENSIVE AND UNIFIED SAFETY EVALUATION BENCHMARK FOR MULTIMODAL LARGE LANGUAGE MODELS (2026.acl-long)
Copied to clipboard
Baolin Zheng, Guanlin Chen, Qingyang Teng, Hongqiong Zhong, Yingshui Tan, Zhendong Liu, Weixun Wang, Jiaheng Liu, Jian Yang, Huiyun Jing, Jincheng Wei, Wenbo Su, Xiaoyong Zhu, Bo Zheng, Kaifu Zhang
| Challenge: | Existing safety benchmarks fail to provide reliable assessments due to limited risk coverage, insufficient scale and the oversight of complex modality combinations. |
| Approach: | They propose a framework that covers 61 risk categories across four modality interactions to address this gap. |
| Outcome: | The proposed framework covers 61 risk categories across four distinct modality interactions. |
CoSafe: Evaluating Large Language Model Safety in Multi-Turn Dialogue Coreference (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have not noticed the safety risks of large language models . authors evaluated 1,400 questions in multi-turn dialogue coreference . |
| Approach: | They are the first to evaluate LLM safety in multi-turn dialogue coreference . they created a dataset of 1,400 questions and tested five open-source models . |
| Outcome: | The study shows that model safety decreases in multi-turn dialogue coreference scenarios . the highest success rate was with the LLaMA2-Chat-7b model, while the lowest was with mistral-7B-Instruct model . |
SURE: Safety Understanding and Reasoning Enhancement for Multimodal Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing multimodal large language models incorporate visual and textual information, but introduces new and complex safety risks. |
| Approach: | They propose a safety reasoning framework that integrates visual modalities into multimodal models to help them resist jailbreak attacks. |
| Outcome: | The proposed framework improves model safety while avoiding over-defense . it is based on a large-scale safety reasoning dataset . |
Can’t See the Forest for the Trees: Benchmarking Multimodal Safety Awareness for Multimodal LLMs (2025.acl-long)
Copied to clipboard
Wenxuan Wang, Xiaoyuan Liu, Kuiyi Gao, Jen-tse Huang, Youliang Yuan, Pinjia He, Shuai Wang, Zhaopeng Tu
| Challenge: | Multimodal Large Language Models (MLLMs) have expanded the capabilities of traditional language models by enabling interaction through both text and images. |
| Approach: | They propose a multimodal safety awareness benchmark to evaluate MLLMs across 29 safety scenarios with 1,500 carefully curated image-prompt pairs. |
| Outcome: | The proposed model is able to identify unsafe content and avoid over-sensitivity that can hinder helpfulness. |
Think in Safety: Unveiling and Mitigating Safety Alignment Collapse in Multimodal Large Reasoning Model (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing Large Reasoning Models have demonstrated broad application potential, yet their safety and reliability remain critical concerns. |
| Approach: | They conduct a safety evaluation of 13 MLRMs across 5 benchmarks and examine their safety performance. |
| Outcome: | The proposed model improves safety on jailbreak and safety-awareness benchmarks. |