Gamma-Guard: Lightweight Residual Adapters for Robust Guardrails in Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) are widely deployed as zero-shot evaluators for answer grading, content moderation, and document ranking. |
| Approach: | They propose a system that trains LLMs with adapters to denoise embeddings and refocus attention. |
| Outcome: | The proposed model lifts adversarial accuracy from 5% to 95% a 90 percentage-point gain while reducing clean-data accuracy by just 8 percentage points. |
Similar Papers
Lightweight Safety Guardrails Using Fine-tuned BERT Embeddings (2025.coling-industry)
Copied to clipboard
| Challenge: | Existing approaches to filter out inappropriate user prompts or system outputs have been successful, but using fine-tuned LLMs as guardrails introduces increased latency and higher maintenance costs. |
| Approach: | They propose to fine-tune a lightweight architecture that reduces the model size from LlamaGuard’s 7 billion parameters to approximately 67 million parameters. |
| Outcome: | Sentence-BERT reduces the model size from 7 billion parameters to approximately 67 million while maintaining comparable performance on the AEGIS safety benchmark. |
LS-Guard: Adaptive Safety Guardrails Tailored to Individual LLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing security guardrails built from static datasets ignore each model’s unique safety profile and often force trade-offs between safety and utility. |
| Approach: | They propose a framework for learning model-specific guardrails tailored to each LLM’s vulnerabilities. |
| Outcome: | The proposed framework significantly outperforms baseline guardrails on multiple real-world LLMs, achieving superior robustness, adaptability, and generalization. |
SELF-GUARD: Empower the LLM to Safeguard Itself (2024.naacl-long)
Copied to clipboard
| Challenge: | Recent studies have investigated methods to improve the safety of large language models (LLMs) safety training involves fine-tuning the LLM with adversarial samples, which activate the LRM’s capabilities against jailbreak. |
| Approach: | They propose a safety training approach that integrates safety training and safeguards to train the LLM to perform harmfulness detection on its own outputs. |
| Outcome: | The proposed method reduces harmful output and adds a [harmful] or [harmless] tag to the end of the LLM's response. |
Why Not Act on What You Know? Unleashing Safety Potential of LLMs via Self-Aware Guard Enhancement (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated impressive capabilities across various tasks but are vulnerable to meticulously crafted jailbreak attacks. |
| Approach: | They propose a training-free defense strategy to align LLMs’ strong safety discrimination performance with their relatively weaker safety generation ability. |
| Outcome: | The proposed strategy achieves an average 99% success rate against numerous complex and covert jailbreak methods while maintaining helpfulness on general benchmarks. |
Safety Guardrails of Large Language Models Are Vulnerable to Value-Driven Adversarial Prompting (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing jailbreak attacks against large language models (LLMs) can be divided into white-box attacks and black-box attack. |
| Approach: | They propose a value-driven jailbreak attack that exploits the phenomenon that large language models agree with humans to induce LLMs to affirm the moral value of harmful tasks. |
| Outcome: | Extensive experiments on five state-of-the-art (SOTA) LLMs show the value-driven jailbreak attack achieves an average attack success rate (ASR) of 91.8% on JailbreakBench and 95.2% on the AdvBench subset. |
Iterative Self-Tuning LLMs for Enhanced Jailbreaking Capabilities (2025.naacl-long)
Copied to clipboard
Chung-En Sun, Xiaodong Liu, Weiwei Yang, Tsui-Wei Weng, Hao Cheng, Aidan San, Michel Galley, Jianfeng Gao
| Challenge: | Recent research shows that Large Language Models (LLMs) are vulnerable to automated jailbreak attacks. |
| Approach: | They propose a framework that crafts adversarial LLMs with enhanced jailbreak ability. |
| Outcome: | ADV-LLM significantly reduces the computational cost of generating adversarial suffixes while achieving nearly 100% ASR on various open-source LLMs. |
Revisiting Jailbreaking for Large Language Models: A Representation Engineering Perspective (2025.coling-main)
Copied to clipboard
Tianlong Li, Zhenghua Wang, Wenhao Liu, Muling Wu, Shihan Dou, Changze Lv, Xiaohua Wang, Xiaoqing Zheng, Xuanjing Huang
| Challenge: | Recent surge in jailbreaking attacks has revealed significant vulnerabilities in Large Language Models (LLMs) however, limited research into the underlying mechanisms that make LLMs vulnerable to such attacks has been conducted. |
| Approach: | They propose that LLMs' self-safeguarding capability is linked to specific activity patterns within their representation space. |
| Outcome: | The proposed models can be detected with a few pairs of contrastive queries, and the robustness can be manipulated by weakening or strengthening these patterns. |
Diversity Helps Jailbreak Large Language Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for jailbreaking large language models rely on laborious human engineering and whitebox access to model internals. |
| Approach: | They propose a method that instructs large language models to deviate from prior context and generate harmful outputs by instructing them to deviat from previous attacks. |
| Outcome: | The proposed method achieves a 62.83% higher success rate in compromising ten leading chatbots, while using only 12.9% of the queries. |
PRP: Propagating Universal Perturbations to Attack Large Language Model Guard-Rails (2024.acl-long)
Copied to clipboard
Neal Mangaokar, Ashish Hooda, Jihye Choi, Shreyas Chandrashekaran, Kassem Fawaz, Somesh Jha, Atul Prakash
| Challenge: | Recent work has shown that large language models are susceptible to automated jailbreak attacks that induce them to generate harmful content. |
| Approach: | They propose a two-step attack strategy that leverages a universal adversarial prefix for the Guard Model and propagates this prefix to the response. |
| Outcome: | The proposed attack strategy is successful against several open-source and closed-source implementations of Guard Models. |
Unleashing the Unseen: Harnessing Benign Datasets for Jailbreaking Large Language Models (2026.findings-eacl)
Copied to clipboard
| Challenge: | Despite significant efforts in safety alignment, large language models (LLMs) such as GPT-4 and LLaMA 3 remain vulnerable to jailbreak attacks that can induce harmful behaviors. |
| Approach: | They propose a feature extraction method to extract sample-agnostic features from benign datasets in the form of adversarial suffixes and propose 'suffix maybe features' they show that adversarials generated from jailbreak attacks may contain meaningful features, i.e. appending the same suffix to different prompts results in responses exhibiting specific characteristics. |
| Outcome: | The proposed method extracts sample-agnostic features from benign datasets and shows that they may contain meaningful features. |