NaturalSloth: Revisiting Denial-of-Service Attacks on Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Longer generations consume more GPU time, increase latency, and reduce throughput in multi-tenant systems. |
| Approach: | They propose an adversarial dataset of natural instruction-based DoS prompts to scale the dataset while preserving malicious intent and increasing semantic diversity. |
| Outcome: | The proposed framework scales with a human-curated seed set of natural instruction-based DoS prompts while preserving malicious intent and increasing semantic diversity. |
Similar Papers
LLMs Caught in the Crossfire: Malware Requests and Jailbreak Challenges (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have a high vulnerability to jailbreak attacks that leverage crafted prompts to generate malicious outputs. |
| Approach: | They propose to use large language models to test their security against jailbreak attacks that leverage crafted prompts to generate malicious outputs. |
| Outcome: | The proposed model is based on 320 manually crafted malicious code generation requirements, covering 11 jailbreak methods and 29 code functionality categories. |
Ignore This Title and HackAPrompt: Exposing Systemic Vulnerabilities of LLMs Through a Global Prompt Hacking Competition (2023.emnlp-main)
Copied to clipboard
Sander Schulhoff, Jeremy Pinto, Anaum Khan, Louis-François Bouchard, Chenglei Si, Svetlina Anati, Valen Tagliabue, Anson Kost, Christopher Carnahan, Jordan Boyd-Graber
| Challenge: | Large Language Models are increasingly being deployed in interactive contexts that involve direct user engagement. |
| Approach: | They run a global prompt hacking competition to encourage research on prompt hacks . they elicit 600K+ adversarial prompts against three state-of-the-art LLMs based on a dataset . |
| Outcome: | The results of the competition show that current LLMs can be manipulated via prompt hacking . the competition elicits 600K+ adversarial prompts against three state-of-the-art LLM models . |
A Wolf in Sheep’s Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easily (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for generating 'jailbreaks' suffer from manual design or require optimization on other white-box models, which compromises either generalization or efficiency. |
| Approach: | They propose a framework that leverages LLMs to generate effective jailbreak prompts and a generalized framework that can be used to generate prompts. |
| Outcome: | The proposed framework improves the attack success rate while reducing the time cost compared to baselines. |
A Comprehensive Study of Jailbreak Attack versus Defense for Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated capabilities for generating content that could be deemed harmful. |
| Approach: | They conduct a comprehensive analysis of existing studies on jailbreaking LLMs and their defense techniques. |
| Outcome: | The proposed techniques underperform existing white-box attacks and include special tokens significantly affects the likelihood of successful attacks. |
Tricking LLMs into Disobedience: Formalizing, Analyzing, and Detecting Jailbreaks (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods to jailbreak large language models have been poorly studied . a recent study showed that non-expert users can jailbreak LLMs by manipulating their prompts . |
| Approach: | They propose a formalism and a taxonomy of known (and possible) jailbreaks . they propose generating a dataset of model outputs across 3700 jailbreak prompts a 'prompt' attack is a new attack popularly categorized as "prompting injection attacks" |
| Outcome: | The proposed model exploits 3700 jailbreak prompts over 4 tasks to analyze their effectiveness . authors show that the model can learn to perform a new task on unseen examples . |
Crabs: Consuming Resource via Auto-generation for LLM-DoS Attack under Black-box Settings (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on white-box attacks focus on black-box LLMs, leaving black- box scenarios underexplored. |
| Approach: | They propose an automated algorithm designed for black-box LLMs that constructs the DoS Attack Tree and expands the node coverage to achieve effectiveness under black- box conditions. |
| Outcome: | The proposed algorithm can be used to build a DoS Attack Tree and expand the node coverage to achieve effectiveness under black-box conditions. |
Diversity Helps Jailbreak Large Language Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for jailbreaking large language models rely on laborious human engineering and whitebox access to model internals. |
| Approach: | They propose a method that instructs large language models to deviate from prior context and generate harmful outputs by instructing them to deviat from previous attacks. |
| Outcome: | The proposed method achieves a 62.83% higher success rate in compromising ten leading chatbots, while using only 12.9% of the queries. |
Attack Prompt Generation for Red Teaming and Defending Large Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies construct attack prompts via manual or automatic methods, but these methods have limitations on cost and quality. |
| Approach: | They propose an attack framework to instruct LLMs to mimic human-generated prompts through in-context learning and a defense framework that fine-tunes victim LLM's through iterative interactions with the attack framework. |
| Outcome: | The proposed approach is based on experiments on different LLMs to evaluate their effectiveness against red teaming attacks. |
Sugar-Coated Poison: Benign Generation Unlocks Jailbreaking (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to jailbreak large language models rely on black-box manipulation of prompt templates, resulting in high costs and poor generalizability. |
| Approach: | They propose a sugar-coated poison attack paradigm that uses a "semantic reversal" strategy to induce the model into a safety response mode. |
| Outcome: | The proposed attack paradigm outperforms baselines in the study. |
Defending Large Language Models Against Jailbreak Attacks via Layer-specific Editing (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing defense methods focus on detecting harmful prompts or reducing the likelihood of harmful responses. |
| Approach: | They propose a layer-specific editing method to align LLMs to harmful prompts by supervised fine-tuning and reinforcement learning. |
| Outcome: | The proposed method improves the performance of large language models against jailbreak attacks while maintaining performance on benign prompts. |