Tianle Gu, Kexin Huang, Zongqi Wang, Yixu Wang, Jie Li, Xin Wang, Yang Yao, Yujiu Yang, Yan Teng, Yingchun Wang
| Challenge: | Despite substantial progress in safety alignment techniques, aligned large language models can still produce unsafe responses under minor internal perturbations. |
| Approach: | They introduce Activation Steering Attack (ASA) and leverage the Negative Log-Likelihood (NLL) as a diagnostic signal to probe the local sensitivity of safety behaviors in latent space. |
| Outcome: | The proposed method is model-agnostic and supervision-free, enabling a general and reproducible diagnostic metric for analyzing safety robustness. |
Similar Papers
Preventing Safety Drift in Large Language Models via Coupled Weight and Activation Constraints (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing defenses constrain either weights or activations in isolation, without considering their coupled effects on safety. |
| Approach: | They propose a weight-activation constraint that enforces a precomputed safety subspace on weight updates and applies regularization to safety-critical features identified by sparse autoencoders. |
| Outcome: | The proposed model outperforms baselines even under high harmful data ratios. |
Why Safeguarded Ships Run Aground? Aligned Large Language Models’ Safety Mechanisms Tend to Be Anchored in The Template Region (2025.acl-long)
Copied to clipboard
| Challenge: | Infilling a fixed template between the input instruction and initial model output is a common practice for existing LLMs, but it is vulnerable to inference-time jailbreak attacks. |
| Approach: | They propose to fill a fixed template between the input instruction and initial model output and to detach safety mechanisms from the template region to mitigate the risk of inference-time jailbreak attacks. |
| Outcome: | The proposed method is widespread across aligned LLMs and shows that it mitigates inference-time jailbreak vulnerabilities. |
How Alignment and Jailbreak Work: Explain LLM Safety through Intermediate Hidden States (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) rely on safety alignment to avoid malicious user inputs. |
| Approach: | They employ weak classifiers to explain LLM safety through the intermediate hidden states. |
| Outcome: | The proposed model can identify malicious and normal inputs and detect malicious ones without jailbreak. |
Unraveling LLM Jailbreaks Through Safety Knowledge Neurons (2026.eacl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have achieved significant progress in alignment, ensuring safer and more reliable outputs. |
| Approach: | They propose a neuron-level interpretability method that focuses on the role of safety-related knowledge neurons to improve model robustness against jailbreak attacks. |
| Outcome: | The proposed method reduces attack success rates across multiple LLMs and outperforms all baseline defenses. |
Analysing the Safety Pitfalls of Steering Vectors (2026.findings-acl)
Copied to clipboard
| Challenge: | Activation steering has emerged as a powerful tool to shape LLM behaviour without the need for weight updates. |
| Approach: | They propose to audit steering vectors obtained with Contrastive Activation Addition (CAA) and propose a mechanistic explanation for this finding. |
| Outcome: | The proposed approach significantly improves the success rate of jailbreak attacks, with stronger amplification under simple template-based attacks. |
Jailbreaks as Inference-Time Alignment: A Framework for Understanding Safety Failures in LLMs (2026.eacl-long)
Copied to clipboard
| Challenge: | Large language models are safety-aligned to prevent harmful response generation . prior work on jailbreak effectiveness has focused on analyzing success rate of jailbreaks . |
| Approach: | They propose to frame jailbreaks as inference-time alignment and draw suboptimal bounds . they also propose a Safety-Net to measure how vulnerable an LLM is to jailbreak attacks . |
| Outcome: | a new framework allows researchers to show how vulnerable an LLM is to jailbreaks . a Safety-Net measures how vulnerable the model is to attacks, the authors say . |
Steering Safely or Off a Cliff? Rethinking Specificity and Robustness in Inference-Time Interventions (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing studies have shown that model steering can preserve fluency and unrelated abilities, but it fails to preserve robustness specificity. |
| Approach: | They propose a framework that distinguishes three dimensions of specificity: general, control, and robustness. |
| Outcome: | The proposed framework distinguishes three dimensions of specificity: general (preserving fluency and unrelated abilities), control (preserving related control properties), and robustness (preserving control properties under distribution shifts). |
Safe-Unsafe Concept Separation Emerges from a Single Direction in Language Models Activation Space (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing approaches to ensuring the safety of Large Language Models (LLMs) rely on invasive fine- tuning or external generation-based checks, which can be opaque and resource-inefficient. |
| Approach: | They propose a mechanistic method that identifies the layer where safe and unsafe concepts are maximally separable within a pretrained representation space. |
| Outcome: | The proposed method can be used across multiple domains, diverse tasks, and 16 non-English languages on encoder and decoder architectures. |
On the Vulnerability of Safety Alignment in Open-Access LLMs (2024.findings-acl)
Copied to clipboard
Jingwei Yi, Rui Ye, Qisi Chen, Bin Zhu, Siheng Chen, Defu Lian, Guangzhong Sun, Xing Xie, Fangzhao Wu
| Challenge: | Large language models (LLMs) are susceptible to malicious exploitation, but are often rejected and limited harmfulness is limited. |
| Approach: | They propose two types of reverse alignment techniques: reverse supervised fine-tuning (RSFT) and reverse preference optimization (RPO). |
| Outcome: | The proposed methods can significantly enhance the success rate and harmfulness of jailbreak attacks, but they face high rejection rates and limited harmfulness. |
Beyond Surface Alignment: Rebuilding LLMs Safety Mechanism via Probabilistically Ablating Refusal Direction (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Jailbreak attacks pose persistent threats to large language models . current safety alignment methods have insufficient safety alignment depth and unrobust internal defense mechanisms. |
| Approach: | a new safety alignment framework is developed to overcome jailbreak attacks . the framework forces the model to dynamically rebuild its refusal mechanisms from jailbreak states . |
| Outcome: | a new safety alignment framework reduces attack success rates by approximately 95% on four open-source LLM families and six representative attacks. |