Ferret: Faster and Effective Automated Red Teaming with Reward-Based Scoring Technique (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing red-teaming methods generate adversarial attacks to identify vulnerabilities, but they face slow performance, limited categorical diversity, and high resource demands. |
| Approach: | They propose a method that generates multiple adversarial prompt mutations per iteration and ranks them using scoring functions. |
| Outcome: | The proposed method achieves a 95% attack success rate and reduces time to a 90% ASR by 15.2%. |
Similar Papers
Automated Progressive Red Teaming (2025.coling-main)
Copied to clipboard
| Challenge: | Automated red teaming (ART) is effective but time-consuming, costly and lacks scalability. |
| Approach: | They propose an automated red teaming framework that generates adversarial prompts to expose LLM vulnerabilities. |
| Outcome: | The proposed framework explores and exploits LLM vulnerabilities through multi-round interactions. |
RedCoder: Automated Multi-Turn Red Teaming for Code LLMs (2026.acl-long)
Copied to clipboard
Wenjie Jacky Mo, Qin Liu, Xiaofei Wen, Dongwon Jung, Hadi Askari, Wenxuan Zhou, Zhe Zhao, Muhao Chen
| Challenge: | Existing red-teaming approaches for code generation rely on extensive human effort and are prone to generating malicious code under adversarial environments. |
| Approach: | They propose a red-teaming agent that engages victim models in multi-turn conversations to elicit vulnerable code. |
| Outcome: | Experiments show that RedCoder outperforms red-teaming methods in inducing vulnerabilities in code generation. |
AART: AI-Assisted Red-Teaming with Diverse Data Generation for New LLM-powered Applications (2023.emnlp-industry)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are rapidly becoming more and more popular, but dealing with the potential harms associated with their deployment in real-world scenarios is still an open research question. |
| Approach: | They propose an automated approach for automated generation of adversarial evaluation datasets to test the safety of LLM generations on new downstream applications. |
| Outcome: | AART generates evaluation datasets with high diversity of content characteristics critical for effective adversarial testing. |
ToxiPrompt: A Two-Stage Red-Teaming Approach for Balancing Adversarial Prompt Diversity and Response Toxicity (2026.eacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) pose safety risks, but current redteaming methods rely on human testers manually designing adversarial prompts. |
| Approach: | They propose a red-teaming method that generates adversarial prompts to elicit unsafe behavior of target LLMs. |
| Outcome: | The proposed approach outperforms state-of-the-art methods in diversity and toxicity . it performs well for multiple instruction-tuned target LLMs without re-tuning . |
Red-Bandit: Test-Time Adaptation for LLM Red-Teaming via Bandit-Guided LoRA Experts (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to audit Large Language Models (LLMs) lack mechanisms to efficiently adapt to model-specific vulnerabilities at inference. |
| Approach: | They propose a red-teaming framework that adapts online to identify and exploit model failure modes under distinct attack styles. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on AdvBench and HarmBench, while generating more human-readable adversarial prompts (lower perplexity). |
Agent vs. Agent: Automated Data Generation and Red-Teaming for Custom Agentic Workflows (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Existing red-teaming frameworks like AgentHarm use static prompts and hardcoded toolsets . |
| Approach: | They propose a red-teaming framework that generates adversarial tasks and evaluation functions tailored to arbitrary toolsets and uses iterative prompt refinement with self-reflection to develop more effective attacks. |
| Outcome: | The proposed approach achieves 162% increase in attack success rate on o4-mini and 86% success on gemini 2.5 Pro. |
MART: Improving LLM Safety with Multi-round Automatic Red-Teaming (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing red-teaming methods for large language models often discover safety risks without addressing them. |
| Approach: | They propose a multi-round automatic red-teaming method that incorporates both adversarial prompt writing and safe response generation. |
| Outcome: | The proposed method significantly increases red-teaming scalability and the safety of the target LLM. |
Holistic Automated Red Teaming for Large Language Models through Top-Down Test Case Generation and Multi-turn Interaction (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches focus on improving attack success rates while overlooking the need for comprehensive test case coverage. |
| Approach: | They propose a top-down approach to automated red teaming that scales up the diversity of test cases using an extensible, fine-grained risk taxonomy. |
| Outcome: | The proposed approach scales up the diversity of test cases using a top-down approach based on an extensible, fine-grained risk taxonomy and leverages reinforcement learning techniques to facilitate multi-turn adversarial probing in a human-like manner. |
Gradient-Based Language Model Red Teaming (2024.eacl-long)
Copied to clipboard
| Challenge: | generative language models generate unsafe responses by producing adversarial prompts . red teaming is labor-intensive and difficult to scale when done by humans. |
| Approach: | They propose a red teaming method that generates diverse prompts that are likely to cause an LM to generate unsafe responses. |
| Outcome: | The proposed method is more effective at finding prompts that trigger an LM to generate unsafe responses than a strong reinforcement learning-based red teaming approach. |
Adaptive Instruction Composition for Automated LLM Red-Teaming (2026.acl-long)
Copied to clipboard
| Challenge: | Adaptive instruction composition is a framework for red-teaming that combines crowdsourced texts with random combinations to optimize effectiveness and diversity. |
| Approach: | They propose a framework that combines crowdsourced texts according to an adaptive mechanism trained to optimize effectiveness with diversity. |
| Outcome: | The proposed framework outperforms random combination on effectiveness and diversity metrics even under model transfer. |