Jailbreak Attack Initializations as Extractors of Compliance Directions (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Safety-aligned LLMs respond to prompts with compliance or refusal, each corresponding to distinct directions in the model’s activation space. |
| Approach: | They propose an initialization framework that aims to project unseen prompts further along compliance directions. |
| Outcome: | The proposed initialization framework achieves an increased attack success rate and reduced computational overhead, highlighting the fragility of safety-aligned LLMs. |
Similar Papers
Beyond Surface Alignment: Rebuilding LLMs Safety Mechanism via Probabilistically Ablating Refusal Direction (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Jailbreak attacks pose persistent threats to large language models . current safety alignment methods have insufficient safety alignment depth and unrobust internal defense mechanisms. |
| Approach: | a new safety alignment framework is developed to overcome jailbreak attacks . the framework forces the model to dynamically rebuild its refusal mechanisms from jailbreak states . |
| Outcome: | a new safety alignment framework reduces attack success rates by approximately 95% on four open-source LLM families and six representative attacks. |
Guiding not Forcing: Enhancing the Transferability of Jailbreaking Attacks on LLMs via Removing Superfluous Constraints (2025.acl-long)
Copied to clipboard
| Challenge: | Jailbreaking attacks can effectively induce unsafe behaviors in Large Language Models (LLMs). |
| Approach: | They propose a conceptual framework to elucidate transferability of gradient-based jailbreaking methods . they identify superfluous constraints as significant barriers to improved transferability . |
| Outcome: | The proposed method increases the overall transfer attack success rate (T-ASR) across target and source models with varying safety levels from 18.4% to 50.3% while improving stability and controllability of jailbreak behaviors. |
Jailbreak LLMs through Internal Stance Manipulation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to exploit LLMs' inherent safety mechanism, including GCG and AutoDAN, are ineffective for certain malicious requests. |
| Approach: | They propose a method that generates jailbreak prompts to suppress a refusal stance and induce affirmative responses by modifying adversarial prompts. |
| Outcome: | The proposed method outperforms the best baseline approach in Llama-2-7b-chat and achieves a 92.2% success rate across all models. |
Activation-Guided Local Editing for Jailbreaking Attacks (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for jailbreaking Large Language Models (LLMs) are limited and produce incoherent or unreadable inputs. |
| Approach: | They propose a two-stage framework that performs a one-shot, scenario-based generation of context and rephrases the original malicious query to obscure its harmful intent. |
| Outcome: | The proposed framework achieves state-of-the-art Attack Success Rate, with gains of up to 37.74% over the strongest baseline, and excellent transferability to black-box and large-scale models. |
Compiling Activation Steering into Weights via Null-Space Constraints for Stealthy Backdoors (2026.acl-long)
Copied to clipboard
Rui Yin, Tianxu Han, Naen Xu, Changjiang Li, Ping He, Chunyi Zhou, Jun Wang, Zhihui Fu, Tianyu Du, Jinbao Li, Shouling Ji
| Challenge: | Existing methods to inject safety-aligned large language models rely on token-level mappings, which do not guarantee sustained harmful output. |
| Approach: | They propose a method that directly modifies model weights to map a trigger to an attacker-specified response. |
| Outcome: | The proposed method achieves high triggered attack success while maintaining non-triggered safety and general utility. |
Shaping the Safety Boundaries: Understanding and Defending Against Jailbreaks in Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Understanding how jailbreaking works remains limited, hindering the development of effective defense strategies. |
| Approach: | They propose a new mechanism that adaptively constrains activations within the safety boundary and propose 'Activation Boundary Defense' to enhance its effectiveness. |
| Outcome: | The proposed defense achieves an average Defense Success Rate (DSR) of over 98% against various jailbreak attacks, with less than 2% impact on the model’s general capabilities. |
Intention Analysis Makes LLMs A Good Jailbreak Defender (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods to align large language models with human values overlook the intrinsic nature of jailbreaks, which limits their effectiveness in complex scenarios. |
| Approach: | They propose a simple yet highly effective defense strategy, i.e., Intention Analysis (IA). They show that IA suppresses LLM’s tendency to follow jailbreak prompts, thereby enhancing safety. |
| Outcome: | The proposed strategy reduces harmfulness of LLMs and outperforms GPT-3.5 in attack success rate. |
Jailbreaks as Inference-Time Alignment: A Framework for Understanding Safety Failures in LLMs (2026.eacl-long)
Copied to clipboard
| Challenge: | Large language models are safety-aligned to prevent harmful response generation . prior work on jailbreak effectiveness has focused on analyzing success rate of jailbreaks . |
| Approach: | They propose to frame jailbreaks as inference-time alignment and draw suboptimal bounds . they also propose a Safety-Net to measure how vulnerable an LLM is to jailbreak attacks . |
| Outcome: | a new framework allows researchers to show how vulnerable an LLM is to jailbreaks . a Safety-Net measures how vulnerable the model is to attacks, the authors say . |
Analysing the Safety Pitfalls of Steering Vectors (2026.findings-acl)
Copied to clipboard
| Challenge: | Activation steering has emerged as a powerful tool to shape LLM behaviour without the need for weight updates. |
| Approach: | They propose to audit steering vectors obtained with Contrastive Activation Addition (CAA) and propose a mechanistic explanation for this finding. |
| Outcome: | The proposed approach significantly improves the success rate of jailbreak attacks, with stronger amplification under simple template-based attacks. |
How Jailbreak Defenses Work and Ensemble? A Mechanistic Investigation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Jailbreak attacks, where harmful prompts bypass generative models’ built-in safety, raise serious concerns about model vulnerability. |
| Approach: | They propose to reframe the standard generation task as a binary classification problem to assess model refusal tendencies for both harmful and benign queries. |
| Outcome: | The proposed defenses improve model safety or optimize the trade-off between safety and helpfulness. |