MART: Improving LLM Safety with Multi-round Automatic Red-Teaming (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing red-teaming methods for large language models often discover safety risks without addressing them. |
| Approach: | They propose a multi-round automatic red-teaming method that incorporates both adversarial prompt writing and safe response generation. |
| Outcome: | The proposed method significantly increases red-teaming scalability and the safety of the target LLM. |
Similar Papers
MTSA: Multi-turn Safety Alignment for LLMs through Multi-round Red-teaming (2025.acl-long)
Copied to clipboard
| Challenge: | Existing jailbreak techniques rely on single-round interactions, pro-Corresponding author. |
| Approach: | They propose a multi-turn safety alignment framework to address the challenge of securing large language models in multi-round interactions. |
| Outcome: | The proposed framework exhibits state-of-the-art attack capabilities while improving safety performance on safety benchmarks. |
Holistic Automated Red Teaming for Large Language Models through Top-Down Test Case Generation and Multi-turn Interaction (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches focus on improving attack success rates while overlooking the need for comprehensive test case coverage. |
| Approach: | They propose a top-down approach to automated red teaming that scales up the diversity of test cases using an extensible, fine-grained risk taxonomy. |
| Outcome: | The proposed approach scales up the diversity of test cases using a top-down approach based on an extensible, fine-grained risk taxonomy and leverages reinforcement learning techniques to facilitate multi-turn adversarial probing in a human-like manner. |
RedCoder: Automated Multi-Turn Red Teaming for Code LLMs (2026.acl-long)
Copied to clipboard
Wenjie Jacky Mo, Qin Liu, Xiaofei Wen, Dongwon Jung, Hadi Askari, Wenxuan Zhou, Zhe Zhao, Muhao Chen
| Challenge: | Existing red-teaming approaches for code generation rely on extensive human effort and are prone to generating malicious code under adversarial environments. |
| Approach: | They propose a red-teaming agent that engages victim models in multi-turn conversations to elicit vulnerable code. |
| Outcome: | Experiments show that RedCoder outperforms red-teaming methods in inducing vulnerabilities in code generation. |
Automated Progressive Red Teaming (2025.coling-main)
Copied to clipboard
| Challenge: | Automated red teaming (ART) is effective but time-consuming, costly and lacks scalability. |
| Approach: | They propose an automated red teaming framework that generates adversarial prompts to expose LLM vulnerabilities. |
| Outcome: | The proposed framework explores and exploits LLM vulnerabilities through multi-round interactions. |
Attack Prompt Generation for Red Teaming and Defending Large Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies construct attack prompts via manual or automatic methods, but these methods have limitations on cost and quality. |
| Approach: | They propose an attack framework to instruct LLMs to mimic human-generated prompts through in-context learning and a defense framework that fine-tunes victim LLM's through iterative interactions with the attack framework. |
| Outcome: | The proposed approach is based on experiments on different LLMs to evaluate their effectiveness against red teaming attacks. |
When Prompt Optimization Becomes Jailbreaking: Adaptive Red-Teaming of Large Language Models (2026.eacl-srw)
Copied to clipboard
| Challenge: | Existing safety evaluations rely on fixed collections of harmful prompts . such attacks span single-shot prompts, multi-turn interactions, cross-lingual settings . |
| Approach: | They propose to use black-box prompt optimization techniques to search for safety failures . they use GPT-5.1 to optimize for a continuous danger score . |
| Outcome: | The proposed approach reduces effective safeguards for large language models . the average danger score of Qwen 3 8B increases from 0.09 in its baseline setting to 0.79 after optimization. |
Gradient-Based Language Model Red Teaming (2024.eacl-long)
Copied to clipboard
| Challenge: | generative language models generate unsafe responses by producing adversarial prompts . red teaming is labor-intensive and difficult to scale when done by humans. |
| Approach: | They propose a red teaming method that generates diverse prompts that are likely to cause an LM to generate unsafe responses. |
| Outcome: | The proposed method is more effective at finding prompts that trigger an LM to generate unsafe responses than a strong reinforcement learning-based red teaming approach. |
Learning to Conceal Risk: Controllable Multi-turn Red Teaming for LLMs in the Financial Domain (2026.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly deployed in high-stakes domains such as finance where unsafe behavior can lead to serious regulatory risks. |
| Approach: | They propose a black-box multi-turn risk-concealed redteaming framework that progressively conceals surface-level risk while exploiting regulatory-violating behaviors. |
| Outcome: | Experiments on nine widely used LLMs show that the proposed framework achieves 93.19% average attack success rate (ASR) and improves the average ASR to 95.00%. |
Be Your Own Red Teamer: Safety Alignment via Self-Play and Reflective Experience Replay (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have remarkable capabilities but are vulnerable to adversarial “jailbreak” attacks designed to bypass safety guardrails. |
| Approach: | They propose to empower a large language model to be its own red teamer . safety self-play allows the model to act as both the Attacker and Defender . |
| Outcome: | The proposed approach outperforms baselines trained on static adversarial datasets and establishes a new benchmark for proactive safety alignment. |
Adaptive Instruction Composition for Automated LLM Red-Teaming (2026.acl-long)
Copied to clipboard
| Challenge: | Adaptive instruction composition is a framework for red-teaming that combines crowdsourced texts with random combinations to optimize effectiveness and diversity. |
| Approach: | They propose a framework that combines crowdsourced texts according to an adaptive mechanism trained to optimize effectiveness with diversity. |
| Outcome: | The proposed framework outperforms random combination on effectiveness and diversity metrics even under model transfer. |