On Guardrail Models’ Robustness to Mutations and Adversarial Attacks (2025.findings-emnlp)
Copied to clipboard
| Challenge: | generative AI systems providing unsafe information has raised significant concerns, emphasizing the need for safety guardrails. |
| Approach: | They propose to evaluate 15 state-of-the-art guardrail models to assess their robustness to input mutations and adversarial attacks designed to bypass models’ safety alignment. |
| Outcome: | The proposed models are robust to input mutations and adversarial attacks that bypass models’ safety alignment. |
Similar Papers
Guardrails and Security for LLMs: Safe, Secure and Controllable Steering of LLM Applications (2025.acl-tutorials)
Copied to clipboard
Traian Rebedea, Leon Derczynski, Shaona Ghosh, Makesh Narsimhan Sreedhar, Faeze Brahman, Liwei Jiang, Bo Li, Yulia Tsvetkov, Christopher Parisien, Yejin Choi
| Challenge: | Pretrained generative models provide novel ways for users to interact with computers. |
| Approach: | This tutorial provides an overview of key guardrail mechanisms developed for LLMs along with evaluation methodologies and a detailed security assessment protocol. |
| Outcome: | This tutorial provides an overview of key guardrail mechanisms developed for LLMs, along with evaluation methodologies and a detailed security assessment protocol. |
Peering Behind the Shield: Guardrail Identification in Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Identifying guardrails in conversational AI agents is critical for identifying malicious content . identifying guardrail components in black-box AI agents poses security challenges . |
| Approach: | They propose a method that leverages guard-specific adversarial prompts to detect guardrails in black-box AI agents. |
| Outcome: | The proposed method achieves perfect classification accuracy in multiple scenarios. |
GuardBench: A Large-Scale Benchmark for Guardrail Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Lack of a standard benchmark for guardrail models poses significant evaluation issues . lack of standardized benchmark makes it hard to compare results across scientific publications. |
| Approach: | They propose a large-scale benchmark for guardrail models comprising 40 safety evaluation datasets. |
| Outcome: | The proposed model achieves competitive results without specific fine-tuning without the need for specific fine tuning. |
LS-Guard: Adaptive Safety Guardrails Tailored to Individual LLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing security guardrails built from static datasets ignore each model’s unique safety profile and often force trade-offs between safety and utility. |
| Approach: | They propose a framework for learning model-specific guardrails tailored to each LLM’s vulnerabilities. |
| Outcome: | The proposed framework significantly outperforms baseline guardrails on multiple real-world LLMs, achieving superior robustness, adaptability, and generalization. |
A Closer Look into the Robustness of Neural Dependency Parsers Using Better Adversarial Examples (2021.findings-acl)
Copied to clipboard
| Challenge: | Neural network-based models have been successful in a wide range of NLP tasks, but their performance is undermined by adversarial examples that would pose no confusion for humans. |
| Approach: | They propose a method to generate high-quality adversarial examples with a higher number of candidate generators and stricter filters and then verify their quality using automatic and human evaluations. |
| Outcome: | The proposed method improves the robustness of English parsing models by relying on adversarial training and model ensembling. |
Gamma-Guard: Lightweight Residual Adapters for Robust Guardrails in Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) are widely deployed as zero-shot evaluators for answer grading, content moderation, and document ranking. |
| Approach: | They propose a system that trains LLMs with adapters to denoise embeddings and refocus attention. |
| Outcome: | The proposed model lifts adversarial accuracy from 5% to 95% a 90 percentage-point gain while reducing clean-data accuracy by just 8 percentage points. |
Is Safer Better? The Impact of Guardrails on the Argumentative Strength of LLMs in Hate Speech Countering (2024.emnlp-main)
Copied to clipboard
| Challenge: | Automated responses lack argumentative richness which characterises expert-produced counterspeech. |
| Approach: | They propose to automate counterspeech generation by investigating tension between helpfulness and harmlessness of LLMs and to assess whether presence of safety guardrails hinders quality of generations. |
| Outcome: | The proposed approach produces more cogent responses that lack argumentative richness which characterises expert-produced counterspeech. |
Robustness and Adversarial Examples in Natural Language Processing (2021.emnlp-tutorials)
Copied to clipboard
| Challenge: | This tutorial aims to raise awareness of practical concerns about NLP robustness . it aims at addressing the weaknesses of NLP systems when faced with adversarial inputs and data with a distribution shift . |
| Approach: | This tutorial aims to bring awareness of practical concerns about NLP robustness . it reviews recent studies on analyzing the weakness of NLP systems when facing adversarial inputs . |
| Outcome: | This tutorial aims to bring awareness of practical concerns about NLP robustness . it will examine the weaknesses of NLP systems when faced with adversarial inputs and data with a distribution shift . |
CodeGuard: Improving LLM Guardrails in CS Education (2026.findings-eacl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are increasingly embedded in Computer Science classrooms to automate code generation, feedback, and assessment. |
| Approach: | They propose a guardrail framework for educational AI systems that can handle unsafe and irrelevant prompts. |
| Outcome: | The proposed framework reduces potentially harmful or policy-violating code completions by 30-65% without degrading performance on legitimate educational tasks. |
Why LLM Safety Guardrails Collapse After Fine-tuning: A Similarity Analysis Between Alignment and Fine-tuning Datasets (2026.acl-long)
Copied to clipboard
| Challenge: | Existing mitigation strategies focus on reactively addressing jailbreak incidents after safety guardrails have been compromised. |
| Approach: | They investigate the degradation of safety guardrails through the lens of representation similarity between upstream alignment datasets and downstream fine-tuning tasks. |
| Outcome: | The proposed model reduces harmfulness score by 10.33% when compared to baseline models. |