Guoyang Zeng, Fanchao Qi, Qianrui Zhou, Tingji Zhang, Zixian Ma, Bairu Hou, Yuan Zang, Zhiyuan Liu, Maosong Sun
| Challenge: | Various attack models are distinct and implemented with different programming frameworks and settings, which hinders quick utilization and fair comparison of attack models. |
| Approach: | They propose an open-source textual adversarial attack toolkit to solve these issues by combining 15 typical attack models into one toolkit. |
| Outcome: | The proposed toolkit supports all attack types, multilinguality, and parallel processing. |
Similar Papers
TextAttack: A Framework for Adversarial Attacks, Data Augmentation, and Adversarial Training in NLP (2020.emnlp-demos)
Copied to clipboard
| Challenge: | TextAttack provides implementations of 16 adversarial attacks from the literature and supports a variety of models and datasets. |
| Approach: | They introduce a Python framework for adversarial attacks, data augmentation, and adversarially training in NLP. |
| Outcome: | This paper introduces a Python framework for adversarial attacks, data augmentation, and adversarially training in NLP. |
Toward Stronger Textual Attack Detectors (2023.findings-emnlp)
Copied to clipboard
| Challenge: | despite the high performance of deep learning techniques, trained models remain vulnerable to adversarial attacks . authors present LAROUSSE, STAKEOUT and other approaches to detect adversarials . LARousSE is unsupervised, hyperparameter free and non-differentiable . |
| Approach: | They propose a framework to detect adversarial attacks and an extended benchmark to test them . they demonstrate that LAROUSSE outperforms previous methods and allows to identify interesting factor of detection rate variations. |
| Outcome: | The proposed framework outperforms existing methods and allows to identify interesting factor of detection rate variations. |
StanceAttack: Adversarial Attack for Stance Detection (2026.findings-acl)
Copied to clipboard
| Challenge: | pretrained language models (PLMs) have greatly enhanced stance detection, but they remain vulnerable to adversarial attacks. |
| Approach: | They propose an adversarial attack method that uses ChatGPT to create adversarials that can mislead well-trained stance detection models. |
| Outcome: | The proposed method outperforms existing adversarial methods with higher success rates and fewer retries on two benchmark datasets. |
LogicAttack: Adversarial Attacks for Evaluating Logical Consistency of Natural Language Inference (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in Large Language Models (LLMs) have demonstrated impressive performance on Natural Language Inference (NLI) tasks. |
| Approach: | They propose a method to attack NLI models using diverse logical forms of premise and hypothesis using propositional logic to generate effective adversarial attacks. |
| Outcome: | The proposed method achieves an average 53% Attack Success Rate (ASR) across multiple logic-based attacks. |
Tougher Text, Smarter Models: Raising the Bar for Adversarial Defence Benchmarks (2025.coling-main)
Copied to clipboard
| Challenge: | Recent advances in natural language processing have highlighted the vulnerability of deep learning models to adversarial attacks. |
| Approach: | They propose a benchmark for textual adversarial defence that evaluates state-of-the-art defence mechanisms across diverse datasets, models, and tasks. |
| Outcome: | The proposed benchmark incorporates a wide range of datasets and evaluates state-of-the-art defence mechanisms. |
Universal Adversarial Attacks with Natural Triggers for Text Classification (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent work has demonstrated the vulnerability of modern text classifiers to universal adversarial attacks, which are input-agnostic sequences of words added to text processed by classifier. |
| Approach: | They propose a gradient-based search that aims to maximize the downstream classifier’s prediction loss by using an adversarially regularized autoencoder to generate triggers and propose heuristics to spot such attacks. |
| Outcome: | The proposed algorithms reduce model accuracy while being less identifiable than prior models as per automatic detection metrics and human-subject studies. |
Word-level Textual Adversarial Attacking as Combinatorial Optimization (2020.acl-main)
Copied to clipboard
| Challenge: | Existing word-level attack models are far from perfect because of unsuitable search space reduction methods and inefficient optimization algorithms. |
| Approach: | They propose a novel adversarial adversarialist model that incorporates word substitution and particle swarm optimization to solve two problems separately. |
| Outcome: | The proposed model achieves much higher success rates and crafts more high-quality adversarial examples as compared to baseline methods. |
Large Language Models Are Better Adversaries: Exploring Generative Clean-Label Backdoor Attacks Against Text Classifiers (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Backdoor attacks manipulate model predictions by inserting malicious "poison" instances that contain a specific pattern or "trigger." |
| Approach: | They propose an attack that inserts style-based triggers into training and test data by using a poison selection technique to improve the effectiveness of both LLMBkd and existing backdoor attacks. |
| Outcome: | The proposed attack achieves high success rates across a wide range of styles with little effort and no model training. |
LPAttack: A Feasible Annotation Scheme for Capturing Logic Pattern of Attacks in Arguments (2022.lrec-1)
Copied to clipboard
| Challenge: | Argumentation plays a central role in human communication, where refuting or attacking others’ arguments is a common persuasion strategy. |
| Approach: | They propose a novel annotation scheme that captures common modes and complex rhetorical moves in attacks along with the implicit presuppositions and value judgments. |
| Outcome: | The proposed scheme shows moderate agreement between the two annotations, indicating that human annotation is feasible. |
Adversarial Attack and Defense of Structured Prediction Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to building effective adversarial attackers focus on classification problems. |
| Approach: | They propose a framework that learns to attack a structured prediction model with feedbacks from multiple reference models. |
| Outcome: | The proposed framework is able to attack state-of-the-art models and boost them with training . it is based on a sequence-to-sequence model with feedbacks from multiple reference models . |