Exploring the Vulnerability of the Content Moderation Guardrail in Large Language Models via Intent Manipulation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Prior work has shown that intent detection enhances LLMs’ moderation guardrails, but the robustness of these guardrail mechanisms under malicious manipulations remains under-explored. |
| Approach: | They propose a two-stage intent-based prompt-refinement framework that first transforms harmful inquiries into structured outlines and further reframes them into declarative-style narratives. |
| Outcome: | The proposed framework outperforms several cutting-edge jailbreak methods and evades even advanced Intent Analysis (IA) and Chain-of-Thought (CoT)-based defenses. |
Similar Papers
Intention Analysis Makes LLMs A Good Jailbreak Defender (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods to align large language models with human values overlook the intrinsic nature of jailbreaks, which limits their effectiveness in complex scenarios. |
| Approach: | They propose a simple yet highly effective defense strategy, i.e., Intention Analysis (IA). They show that IA suppresses LLM’s tendency to follow jailbreak prompts, thereby enhancing safety. |
| Outcome: | The proposed strategy reduces harmfulness of LLMs and outperforms GPT-3.5 in attack success rate. |
Distract Large Language Models for Automatic Jailbreak Attack (2024.emnlp-main)
Copied to clipboard
| Challenge: | Commercial large language models (LLMs) have made great progress in various NLP tasks. |
| Approach: | They propose a black-box jailbreak framework for automated red teaming of Large language models using an iterative optimization algorithm to conceal malicious content and memory reframing. |
| Outcome: | The proposed framework outperforms existing jailbreak defense methods and highlights the need to develop more effective and practical defense strategies. |
Tricking LLMs into Disobedience: Formalizing, Analyzing, and Detecting Jailbreaks (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods to jailbreak large language models have been poorly studied . a recent study showed that non-expert users can jailbreak LLMs by manipulating their prompts . |
| Approach: | They propose a formalism and a taxonomy of known (and possible) jailbreaks . they propose generating a dataset of model outputs across 3700 jailbreak prompts a 'prompt' attack is a new attack popularly categorized as "prompting injection attacks" |
| Outcome: | The proposed model exploits 3700 jailbreak prompts over 4 tasks to analyze their effectiveness . authors show that the model can learn to perform a new task on unseen examples . |
Exploring Jailbreak Attacks on LLMs through Intent Concealment and Diversion (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing jailbreak methods face an excessive number of iterative queries and poor generalization across models. |
| Approach: | They propose a jailbreak method that employs **I**ntent **C**oncealment and div**E**rsion to circumvent security constraints. |
| Outcome: | The proposed method outperforms existing jailbreak techniques in question-answering and text-generation tasks. |
Chain of Attack: Hide Your Intention through Multi-Turn Interrogation (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing jailbreak attacks focus on single-turn dialogue scenarios, leaving vulnerabilities in multi-turn contexts inadequately explored. |
| Approach: | They propose an optimal interrogation principle to conceal the jailbreak intent and introduce a multi-turn attack chain generation strategy called CoA. |
| Outcome: | The proposed method shows that black-box LLMs exhibit insufficient resistance under multi-turn interrogation, with more advantages (ASR, 83% vs 64%) |
Shaping the Safety Boundaries: Understanding and Defending Against Jailbreaks in Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Understanding how jailbreaking works remains limited, hindering the development of effective defense strategies. |
| Approach: | They propose a new mechanism that adaptively constrains activations within the safety boundary and propose 'Activation Boundary Defense' to enhance its effectiveness. |
| Outcome: | The proposed defense achieves an average Defense Success Rate (DSR) of over 98% against various jailbreak attacks, with less than 2% impact on the model’s general capabilities. |
Beyond Surface-Level Detection: Towards Cognitive-Driven Defense Against Jailbreak Attacks via Meta-Operations Reasoning (2026.acl-long)
Copied to clipboard
| Challenge: | Existing defenses rely on shallow pattern matching, which struggles to generalize to novel and unseen attack strategies. |
| Approach: | They propose a framework which emulates human cognitive reasoning through a structured reasoning chain. |
| Outcome: | The proposed framework achieves state-of-the-art performance and exhibits strong generalization to unseen attacks. |
Understanding Jailbreak Success: A Study of Latent Space Dynamics in Large Language Models (2026.eacl-long)
Copied to clipboard
| Challenge: | Emerging jailbreaking techniques can still elicit unsafe outputs, presenting an ongoing challenge for model alignment. |
| Approach: | They propose to extract a jailbreak vector from a single class of jailbreaks that works to mitigate jailbreak effectiveness from other, semantically-dissimilar classes. |
| Outcome: | The proposed jailbreak vectors show that they reduce harmfulness in most models, and that they are similar in geometric and effect similarity. |
Revisiting Jailbreaking for Large Language Models: A Representation Engineering Perspective (2025.coling-main)
Copied to clipboard
Tianlong Li, Zhenghua Wang, Wenhao Liu, Muling Wu, Shihan Dou, Changze Lv, Xiaohua Wang, Xiaoqing Zheng, Xuanjing Huang
| Challenge: | Recent surge in jailbreaking attacks has revealed significant vulnerabilities in Large Language Models (LLMs) however, limited research into the underlying mechanisms that make LLMs vulnerable to such attacks has been conducted. |
| Approach: | They propose that LLMs' self-safeguarding capability is linked to specific activity patterns within their representation space. |
| Outcome: | The proposed models can be detected with a few pairs of contrastive queries, and the robustness can be manipulated by weakening or strengthening these patterns. |
Towards Understanding Jailbreak Attacks in LLMs: A Representation Space Analysis (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) are susceptible to a type of attack known as jailbreaking, which misleads LLMs to output harmful contents. |
| Approach: | They propose to leverage hidden representations into existing jailbreak targets to move the attacks along the acceptance direction. |
| Outcome: | The proposed methods are validated using the objective of existing jailbreak attacks. |