From Insight to Exploit: Leveraging LLM Collaboration for Adaptive Adversarial Text Generation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | LLMs can provide substantial zero-shot performance on diverse tasks, but it is crucial to assess their robustness against adversarial inputs. |
| Approach: | They introduce Static Deceptor and Dynamic Deceptr to generate adversarial examples . they produce subtle and natural-looking adversarials that preserve semantic similarity to text . |
| Outcome: | The proposed attacks are based on two LLM-based attacks that generate natural-looking examples that deceive the target LLM. |
Similar Papers
Neuronal Insights into LLM Attacks: Targeted Neuron Tuning for Precise and Robust Vulnerability Patching (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing gradient-based attribution methods are inapplicable to adversarial attacks . et al.: Targeted neuron tuning improves model robustness against jailbreak attacks despite the model's vulnerability to jailbreak. |
| Approach: | They propose a gradient-based method to identify key neurons sensitive to adversarial behaviors in open-ended generation tasks. |
| Outcome: | The proposed method detects key neurons sensitive to adversarial behaviors in open-ended tasks. |
Is LLM-as-a-Judge Robust? Investigating Universal Adversarial Attacks on Zero-shot LLM Assessment (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are powerful zero-shot assessors used in real-world situations . however, no study has examined the vulnerability of judge-LLM to adversarial manipulation . |
| Approach: | They propose a simple surrogate attack where a surrogated model is attacked and the learned attack phrase transferred to unknown judge-LLMs. |
| Outcome: | The proposed algorithm shows that judge-LLMs can be significantly more susceptible to adversarial attacks when used for absolute scoring, rather than comparative assessment. |
MultiAgent Collaboration Attack: Investigating Adversarial Attacks in Large Language Model Collaborations via Debate (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown exceptional results when working individually, and have reduced parameter size and inference times. |
| Approach: | They evaluate the behavior of a network of models collaborating through debate under the influence of an adversary and examine inference-time methods to generate more compelling arguments. |
| Outcome: | The proposed model-based model-driven analysis shows that the model-led model-mediated debates generate more compelling arguments and provide a defensive strategy. |
Close or Cloze? Assessing the Robustness of Large Language Models to Adversarial Perturbations via Word Recovery (2025.coling-main)
Copied to clipboard
| Challenge: | Existing models implicitly recover the original text, but it is unclear when they rely on context and when they implicitly do so. |
| Approach: | They propose to use a dictionary to recover adversarial words by using a phonetic, typo, and visual attack to study word recovery performance. |
| Outcome: | The proposed model outperforms open-source models on hateful, offensive, and toxic classification tasks. |
Vulnerabilities of Large Language Models to Adversarial Attacks (2024.acl-tutorials)
Copied to clipboard
| Challenge: | This tutorial focuses on the vulnerabilities of Large Language Models to adversarial attacks . the tutorial lays the foundation by explaining safety-aligned models and concepts in cybersecurity . |
| Approach: | This tutorial lays the foundation by explaining safety-aligned LLMs and concepts in cybersecurity. |
| Outcome: | The tutorial lays the foundation by explaining safety-aligned models and concepts in cybersecurity. |
Automated Progressive Red Teaming (2025.coling-main)
Copied to clipboard
| Challenge: | Automated red teaming (ART) is effective but time-consuming, costly and lacks scalability. |
| Approach: | They propose an automated red teaming framework that generates adversarial prompts to expose LLM vulnerabilities. |
| Outcome: | The proposed framework explores and exploits LLM vulnerabilities through multi-round interactions. |
Humanizing Machine-Generated Content: Evading AI-Text Detection through Adversarial Attack (2024.lrec-main)
Copied to clipboard
| Challenge: | Despite the development of large language models, there are still significant challenges in detecting whether text is generated by a machine. |
| Approach: | They propose a framework for a broader class of adversarial attacks to perform minor perturbations in machine-generated content to evade detection. |
| Outcome: | The proposed framework can be compromised in as little as 10 seconds, and improves over iterative adversarial learning. |
Adaptive Detoxification: Safeguarding General Capabilities of LLMs through Toxicity-Aware Knowledge Editing (2025.findings-acl)
Copied to clipboard
Yifan Lu, Jing Li, Yigeng Zhou, Yihui Zhang, Wenya Wang, Xiucheng Li, Meishan Zhang, Fangming Liu, Jun Yu, Min Zhang
| Challenge: | Existing knowledge editing methods for large language models (LLMs) suffer from over-editing, where detoxified models reject legitimate queries, compromising overall performance. |
| Approach: | They propose a toxicity-aware knowledge editing approach that dynamically detects toxic activation patterns during forward propagation and then routes computations through adaptive inter-layer pathways to mitigate toxicity effectively. |
| Outcome: | The proposed method outperforms existing methods on large language models and enhances the SafeEdit benchmark. |
Adaptive Instruction Composition for Automated LLM Red-Teaming (2026.acl-long)
Copied to clipboard
| Challenge: | Adaptive instruction composition is a framework for red-teaming that combines crowdsourced texts with random combinations to optimize effectiveness and diversity. |
| Approach: | They propose a framework that combines crowdsourced texts according to an adaptive mechanism trained to optimize effectiveness with diversity. |
| Outcome: | The proposed framework outperforms random combination on effectiveness and diversity metrics even under model transfer. |
How Trustworthy are Open-Source LLMs? An Assessment under Malicious Demonstrations Shows their Vulnerabilities (2024.naacl-long)
Copied to clipboard
| Challenge: | Rapid progress in open-source Large Language Models (LLMs) is driving AI development, but lacks sufficient trustworthiness to detect and mitigate adversarial demonstrations. |
| Approach: | They propose an extended Chain of Utterances-based (CoU) prompting strategy to attack open-source LLMs. |
| Outcome: | The proposed attack strategy is based on malicious demonstrations and toxicity tests on open-source models. |