How Jailbreak Defenses Work and Ensemble? A Mechanistic Investigation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Jailbreak attacks, where harmful prompts bypass generative models’ built-in safety, raise serious concerns about model vulnerability. |
| Approach: | They propose to reframe the standard generation task as a binary classification problem to assess model refusal tendencies for both harmful and benign queries. |
| Outcome: | The proposed defenses improve model safety or optimize the trade-off between safety and helpfulness. |
Similar Papers
Shaping the Safety Boundaries: Understanding and Defending Against Jailbreaks in Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Understanding how jailbreaking works remains limited, hindering the development of effective defense strategies. |
| Approach: | They propose a new mechanism that adaptively constrains activations within the safety boundary and propose 'Activation Boundary Defense' to enhance its effectiveness. |
| Outcome: | The proposed defense achieves an average Defense Success Rate (DSR) of over 98% against various jailbreak attacks, with less than 2% impact on the model’s general capabilities. |
CAVGAN: Unifying Jailbreak and Defense of LLMs via Generative Adversarial Attacks on their Internal Representations (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing studies have isolated LLM jailbreak attacks and defenses . a new framework combines attack and defense to protect against malicious queries . |
| Approach: | They propose a framework that combines attack and defense to protect the Large Language Model (LLM) by embedding harmful problems into the safe area. |
| Outcome: | The proposed framework achieves an average jailbreak success rate of 88.85% across three popular LLMs while the defense success rate reaches an average of 84.17%. |
How Alignment and Jailbreak Work: Explain LLM Safety through Intermediate Hidden States (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) rely on safety alignment to avoid malicious user inputs. |
| Approach: | They employ weak classifiers to explain LLM safety through the intermediate hidden states. |
| Outcome: | The proposed model can identify malicious and normal inputs and detect malicious ones without jailbreak. |
Understanding Jailbreak Success: A Study of Latent Space Dynamics in Large Language Models (2026.eacl-long)
Copied to clipboard
| Challenge: | Emerging jailbreaking techniques can still elicit unsafe outputs, presenting an ongoing challenge for model alignment. |
| Approach: | They propose to extract a jailbreak vector from a single class of jailbreaks that works to mitigate jailbreak effectiveness from other, semantically-dissimilar classes. |
| Outcome: | The proposed jailbreak vectors show that they reduce harmfulness in most models, and that they are similar in geometric and effect similarity. |
Towards Understanding Jailbreak Attacks in LLMs: A Representation Space Analysis (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) are susceptible to a type of attack known as jailbreaking, which misleads LLMs to output harmful contents. |
| Approach: | They propose to leverage hidden representations into existing jailbreak targets to move the attacks along the acceptance direction. |
| Outcome: | The proposed methods are validated using the objective of existing jailbreak attacks. |
Tricking LLMs into Disobedience: Formalizing, Analyzing, and Detecting Jailbreaks (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods to jailbreak large language models have been poorly studied . a recent study showed that non-expert users can jailbreak LLMs by manipulating their prompts . |
| Approach: | They propose a formalism and a taxonomy of known (and possible) jailbreaks . they propose generating a dataset of model outputs across 3700 jailbreak prompts a 'prompt' attack is a new attack popularly categorized as "prompting injection attacks" |
| Outcome: | The proposed model exploits 3700 jailbreak prompts over 4 tasks to analyze their effectiveness . authors show that the model can learn to perform a new task on unseen examples . |
Exploring Jailbreak Attacks on LLMs through Intent Concealment and Diversion (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing jailbreak methods face an excessive number of iterative queries and poor generalization across models. |
| Approach: | They propose a jailbreak method that employs **I**ntent **C**oncealment and div**E**rsion to circumvent security constraints. |
| Outcome: | The proposed method outperforms existing jailbreak techniques in question-answering and text-generation tasks. |
SafeInt: Shielding Large Language Models from Jailbreak Attacks via Safety-Aware Representation Intervention (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Jailbreak attacks exploit vulnerabilities in large language models to induce undesirable behavior . existing defenses cannot dynamically adjust representations based on harmfulness of queries . |
| Approach: | They propose a representation-aware representation method that shields LLMs from jailbreak attacks . SafeInt relocates jailbreak-related representations into the rejection region . |
| Outcome: | The proposed method outperforms baseline defenses while maintaining utility . it relocates jailbreak-related representations into the rejection region . |
From Attack Surfaces to Actual Operations: A Survey of Modern LLM Jailbreaks (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing taxonomies focus on manipulation methods rather than underlying mechanisms, limiting our understanding of attack effectiveness and defensive strategies. |
| Approach: | They propose a two-fold taxonomy to categorize attacks across three tiers based on exploited vulnerabilities and approaches and an operational taxonomies to evaluate attacks across four dimensions. |
| Outcome: | The proposed taxonomy categorizes attacks across three tiers based on exploited vulnerabilities and approaches and evaluates attacks on four dimensions to assess real-world feasibility and sustainability. |
Jailbreaking Attacks vs. Content Safety Filters: How Far Are We in the LLM Safety Arms Race? (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing studies have focused on the models, neglecting the full deployment pipeline . previous studies have underestimated the practical success of these attacks . |
| Approach: | They evaluate the effectiveness of jailbreak attacks targeting LLM safety alignment . they highlight critical gaps and call for further refinement of detection accuracy and usability . |
| Outcome: | The proposed attacks can detect at least one safety filter across the entire deployment pipeline. |