On the Vulnerability of Safety Alignment in Open-Access LLMs (2024.findings-acl)
Copied to clipboard
Jingwei Yi, Rui Ye, Qisi Chen, Bin Zhu, Siheng Chen, Defu Lian, Guangzhong Sun, Xing Xie, Fangzhao Wu
| Challenge: | Large language models (LLMs) are susceptible to malicious exploitation, but are often rejected and limited harmfulness is limited. |
| Approach: | They propose two types of reverse alignment techniques: reverse supervised fine-tuning (RSFT) and reverse preference optimization (RPO). |
| Outcome: | The proposed methods can significantly enhance the success rate and harmfulness of jailbreak attacks, but they face high rejection rates and limited harmfulness. |
Similar Papers
The Art of (Mis)alignment: How Fine-Tuning Methods Effectively Misalign and Realign LLMs in Post-Training (2026.findings-acl)
Copied to clipboard
Rui Zhang, Hongwei Li, Yun Shen, Xinyue Shen, Wenbo Jiang, Guowen Xu, Yang Liu, Michael Backes, Yang Zhang
| Challenge: | Misaligned large language models can magnify harm by exploiting them to undermine safety . et al., 2022b; Bai e.t., 2023): misalignment, realignment and model-specific resistance are important . |
| Approach: | They evaluate four methods to identify a mechanism asymmetry between attack and defense . they find that ORPO is most effective for misalignment, but DPO excels in realignment . |
| Outcome: | The proposed methods show a mechanism asymmetry between attack and defense . the proposed methods excel in realignment, but at the expense of model utility . |
Defending Against Alignment-Breaking Attacks via Robustly Aligned LLM (2024.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have made significant advancements but can be misused to generate harmful content. |
| Approach: | They propose a Robustly Aligned LLM to defend against alignment-breaking attacks by retraining existing LLMs and using adversarial or handcrafted jailbreaking prompts. |
| Outcome: | The proposed model reduces attack success rates from nearly 100% to around 10% or less. |
How Alignment and Jailbreak Work: Explain LLM Safety through Intermediate Hidden States (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) rely on safety alignment to avoid malicious user inputs. |
| Approach: | They employ weak classifiers to explain LLM safety through the intermediate hidden states. |
| Outcome: | The proposed model can identify malicious and normal inputs and detect malicious ones without jailbreak. |
Separate the Wheat from the Chaff: A Post-Hoc Approach to Safety Re-Alignment for Fine-Tuned Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models achieve effective safety alignment at the time of release, but fine-tuning often compromises safety mechanisms. |
| Approach: | They propose a method that performs safety realignment for large language models . they identify unsafe delta parameters from the fine-tuned models and recalibrate the retained parameters . |
| Outcome: | The proposed method improves safety performance on safety benchmarks and jailbreak attacks while maintaining their performance on downstream tasks. |
Towards Understanding the Fragility of Multilingual LLMs against Fine-Tuning Attacks (2025.findings-naacl)
Copied to clipboard
| Challenge: | Recent advances in Large Language Models have sparked concerns about their safety. |
| Approach: | They propose a method to identify safety-related information in the model parameter space . they propose to use a few adversarially chosen examples to fine-tune LLMs . |
| Outcome: | The proposed method can break safety alignment in multilingual LLMs using a few examples . it also shows that the proposed method jailbreaks LLM models adapted to new languages . |
Unraveling LLM Jailbreaks Through Safety Knowledge Neurons (2026.eacl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have achieved significant progress in alignment, ensuring safer and more reliable outputs. |
| Approach: | They propose a neuron-level interpretability method that focuses on the role of safety-related knowledge neurons to improve model robustness against jailbreak attacks. |
| Outcome: | The proposed method reduces attack success rates across multiple LLMs and outperforms all baseline defenses. |
Multitask-Bench: Unveiling and Mitigating Safety Gaps in LLMs Fine-tuning (2025.coling-main)
Copied to clipboard
| Challenge: | Recent advances in Large Language Models (LLMs) have led to their adoption across a wide range of tasks, ranging from code generation to machine translation and sentiment analysis. |
| Approach: | They propose to fine-tune LLMs on benign (non-harmful) data to ensure safe outputs. |
| Outcome: | The proposed model reduces attack success rates across a range of tasks without compromising its usefulness. |
SABER: Uncovering Vulnerabilities in Safety Alignment via Cross-Layer Residual Connection (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) with safe-alignment training are vulnerable to jailbreak attacks, causing malicious users to generate harmful outputs. |
| Approach: | They propose a safe-alignment jailbreak method that bypasses the middle-to-late layers of large language models by a residual connection. |
| Outcome: | The proposed method improves by 51% over the best performing baseline GCG on HarmBench test set. |
Jailbreaks as Inference-Time Alignment: A Framework for Understanding Safety Failures in LLMs (2026.eacl-long)
Copied to clipboard
| Challenge: | Large language models are safety-aligned to prevent harmful response generation . prior work on jailbreak effectiveness has focused on analyzing success rate of jailbreaks . |
| Approach: | They propose to frame jailbreaks as inference-time alignment and draw suboptimal bounds . they also propose a Safety-Net to measure how vulnerable an LLM is to jailbreak attacks . |
| Outcome: | a new framework allows researchers to show how vulnerable an LLM is to jailbreaks . a Safety-Net measures how vulnerable the model is to attacks, the authors say . |
Towards Understanding Jailbreak Attacks in LLMs: A Representation Space Analysis (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) are susceptible to a type of attack known as jailbreaking, which misleads LLMs to output harmful contents. |
| Approach: | They propose to leverage hidden representations into existing jailbreak targets to move the attacks along the acceptance direction. |
| Outcome: | The proposed methods are validated using the objective of existing jailbreak attacks. |