Jailbreaking Safeguarded Text-to-Image Models via Large Language Models (2026.findings-eacl)
Copied to clipboard
| Challenge: | Text-to-image models generate harmful content when unsafe prompts are submitted . authors propose a method to jailbreak text-to image models with safety guardrails . |
| Approach: | They propose a method to jailbreak text-to-image models with safety guardrails . they use a fine-tuned large language model to generate adversarial prompts based on unsafe prompts. |
| Outcome: | The proposed method bypasses safety guardrails and outperforms existing no-box attacks . the proposed method generates adversarial prompts efficiently after fine-tuning the model . |
Similar Papers
Jailbreaking Prompt Attack: A Controllable Adversarial Attack against Diffusion Models (2025.findings-naacl)
Copied to clipboard
| Challenge: | Text-to-image (T2I) models can be used to generate harmful content such as sexually explicit, unfaithful, and misleading or Not-Safe-for-Work (NSFW) images. |
| Approach: | They propose a more practical and universal attack that does not require the presence of a target model. |
| Outcome: | The proposed attack bypasses both text and image safety checkers while preserving high semantic alignment with the target prompt. |
Multimodal Pragmatic Jailbreak on Text-to-image Models (2025.acl-long)
Copied to clipboard
Tong Liu, Zhixin Lai, Jiawen Wang, Gengyuan Zhang, Shuo Chen, Philip Torr, Vera Demberg, Volker Tresp, Jindong Gu
| Challenge: | Existing jailbreaks for diffusion-based text-to-image models generate unsafe content . experimental results show that all tested models suffer from unsafe generation . |
| Approach: | They propose a jailbreak that triggers diffusion-based text-to-image models to generate the image with visual text, resulting in unsafe content. |
| Outcome: | The proposed model generates image with visual text, but the model is unsafe under such jailbreak. |
Seeing No Evil: Blinding Large Vision-Language Models to Safety Instructions via Adversarial Attention Hijacking (2026.acl-long)
Copied to clipboard
| Challenge: | Existing attacks optimize image perturbations to maximize harmful output likelihood, but suffer from slow convergence due to gradient conflict between adversarial objectives and the model’s safety-retrieval mechanism. |
| Approach: | They propose a push-pull approach which suppresses attention to system-prompt tokens and anchors generation on adversarial image features to avoid collisions. |
| Outcome: | The proposed approach reduces gradient conflict by 45% and achieves 94.4% attack success rate on Qwen-VL (vs. 68.8% baseline) with 40% fewer iterations. |
Safety Guardrails of Large Language Models Are Vulnerable to Value-Driven Adversarial Prompting (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing jailbreak attacks against large language models (LLMs) can be divided into white-box attacks and black-box attack. |
| Approach: | They propose a value-driven jailbreak attack that exploits the phenomenon that large language models agree with humans to induce LLMs to affirm the moral value of harmful tasks. |
| Outcome: | Extensive experiments on five state-of-the-art (SOTA) LLMs show the value-driven jailbreak attack achieves an average attack success rate (ASR) of 91.8% on JailbreakBench and 95.2% on the AdvBench subset. |
Iterative Prompt Refinement for Safer Text-to-Image Generation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing safety methods for text-to-image models ignore the images produced . this can result in unsafe outputs or unnecessary changes to already safe prompts . |
| Approach: | They propose an iterative prompt refinement algorithm that uses Vision Language Models to analyze prompts and generated images. |
| Outcome: | The proposed method improves safety while maintaining user intent and reliability comparable to existing methods. |
Chain-of-Jailbreak Attack for Image Generation Models via Step by Step Editing (2025.findings-acl)
Copied to clipboard
Wenxuan Wang, Kuiyi Gao, Youliang Yuan, Jen-tse Huang, Qiuzhi Liu, Shuai Wang, Wenxiang Jiao, Zhaopeng Tu
| Challenge: | Text-based image generation models, such as Stable Diffusion and DALL-E 3, hold significant potential in content creation and publishing workflows . however, considerable efforts are being made to prevent the generation of harmful content, such abusive, violent, or pornographic material. |
| Approach: | They propose a chain-of-jailbreak method which decomposes malicious queries into multiple sub-queries and iteratively edits images based on these sub-questions. |
| Outcome: | The proposed method can bypass safeguards of image generation models for over 60% cases, significantly outperforms other jailbreaking methods (14%) |
SAFENUDGE: Safeguarding Large Language Models in Real-time with Tunable Safety-Performance Trade-offs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are susceptible to jailbreak attacks, or adversarial prompts eliciting high-risk behavior. |
| Approach: | They propose a safeguard that combines Controlled Text Generation and "nudging" it adds minimal latency to inference and reduces successful jailbreak attempts by up to 37.3% . |
| Outcome: | The proposed safeguard reduces successful jailbreak attempts by between 28.1% and 37.3% by guiding the LLM towards a safe response. |
Stealthy Jailbreak Attacks on Large Language Models via Benign Data Mirroring (2025.naacl-long)
Copied to clipboard
Honglin Mu, Han He, Yuxin Zhou, Yunlong Feng, Yang Xu, Libo Qin, Xiaoming Shi, Zeming Liu, Xudong Han, Qi Shi, Qingfu Zhu, Wanxiang Che
| Challenge: | Existing black-box jailbreak methods often rely on model feedback . existing methods may be intercepted by content moderators during the search process . |
| Approach: | They propose a method that guides malicious prompt construction by local training a mirror model of the target black-box model through benign data distillation. |
| Outcome: | The proposed method achieves a 92% attack success rate and 80% stealth rate on a subset of AdvBench. |
Defensive Prompt Patch: A Robust and Generalizable Defense of Large Language Models against Jailbreak Attacks (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have showcased their ability to understand and generate text akin to human interaction. |
| Approach: | They propose a prompt-based defense mechanism specifically designed to protect LLMs against jailbreak attacks by introducing jailbreak prompts into malicious queries. |
| Outcome: | Empirical results show that the proposed defense outperforms existing defense strategies in balancing safety and utility while maintaining high utility. |
SequentialBreak: Large Language Models Can be Fooled by Embedding Jailbreak Prompts into Sequential Prompt Chains (2025.acl-srw)
Copied to clipboard
| Challenge: | SequentialBreak enables LLMs to bypass safety mechanisms by arranging malicious prompts in a single query. |
| Approach: | They propose a single-query jailbreak technique that arranges multiple benign prompts in sequence with a hidden malicious instruction among them to bypass safety mechanisms. |
| Outcome: | The proposed technique outperforms baselines on open-source and closed-source models. |