A Black-Box Attack on Code Models via Representation Nearest Neighbor Search (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for generating adversarial code examples face challenges such as limted availability of substitute variables and the creation of adversarials with noticeable perturbations. |
| Approach: | They propose a search seed based on historical attacks to find adversarial substitutes . they employ a pre-trained variable name encoder to map the search seed to a continuous vector space . |
| Outcome: | The proposed approach outperforms baseline methods in terms of ASR and QT. |
Similar Papers
Query-Efficient Textual Adversarial Example Generation for Black-Box Attacks (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing black-box attacks require thousands of queries on the target model, making them expensive in real-world applications. |
| Approach: | They propose a new approach that guides word substitutions using prior knowledge from the training set to improve the attack efficiency. |
| Outcome: | The proposed approach reduces query-free attack and guided search attacks by a factor of 10 500 . it improves transferability and generalization by the ensemble of the ABPens in NLP . |
TABS: Efficient Textual Adversarial Attack for Pre-trained NL Code Model Using Semantic Beam Search (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing black-box adversarial attacks on pre-trained models generate adversarials with greedy search. |
| Approach: | They propose an efficient beam search black-box adversarial attack method . they use contextual semantic filtering to effectively reduce the search space . |
| Outcome: | The proposed method shows good performance in terms of attack success rate, number of queries, and semantic similarity for two tasks: NL code search classification and retrieval tasks. |
SHIELD: Defending Textual Neural Networks against Multiple Black-Box Adversarial Attacks with Stochastic Multi-Expert Patcher (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods to defend textual neural network models against adversarial attacks often require retraining and retrain . e.g., BERT, RoBERTa require great time and computation resources. |
| Approach: | They propose an algorithm that modifies and re-trains only the last layer of a textual NN and transforms it into a stochastic weighted ensemble of multi-expert prediction heads. |
| Outcome: | The proposed algorithm outperforms existing models against black-box attacks by 15%–70% . the proposed algorithm is based on a novel algorithm from software engineering . |
DIP: Dead code Insertion based Black-box Attack for Programming Language Model (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods to attack natural language models are difficult to apply due to the requirements. |
| Approach: | They propose a black-box attack method that generates adversarial examples using dead code insertion. |
| Outcome: | The proposed method outperforms the state-of-the-art black-box attack in both attack efficiency and attack quality on 9 victim downstream-task large code models. |
Contrastive Code Representation Learning (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent work learns contextual representations of source code by reconstructing tokens from their context. |
| Approach: | They propose a contrastive pre-training task that learns code functionality, not form . they propose scalable compilers that can generate variants of a program . |
| Outcome: | The proposed task outperforms RoBERTa on an adversarial code clone detection benchmark by 39% AUROC. |
Improving Gradient-based Adversarial Training for Text Classification by Contrastive Learning and Auto-Encoder (2021.findings-acl)
Copied to clipboard
| Challenge: | Recent work has shown that models can be easily fooled by intentionally designed adversarial examples. |
| Approach: | They propose two efficient approaches for generating adversarial perturbations on embeddings and propose two new approaches to help model learn adversarials more efficiently. |
| Outcome: | The proposed approaches outperform strong baselines on various text classification datasets and the model's performance drops less under adversarial attack. |
White-to-Black: Efficient Distillation of Black-Box Adversarial Attacks (N19-1)
Copied to clipboard
| Challenge: | Recent work in natural language processing generates adversarial examples using white-box access . a neural network can learn to emulate the behavior of a white- box attack and generalize well to new examples. |
| Approach: | They propose an adversarial training approach that assumes white-box access to an attacker's model and optimizes the input directly against it. |
| Outcome: | The proposed approach reduces example generation time by 19x-39x and exposes the Google Perspective API vulnerability. |
T3: Tree-Autoencoder Constrained Adversarial Text Generation for Targeted Attack (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing adversarial examples can induce arbitrary errors to the target models, but they can be exploited to estimate robustness of NLP models. |
| Approach: | They propose a target-controllable adversarial attack framework T3 to handle adversarials . they use tree-based decoders to regularize the syntactic correctness of generated text . |
| Outcome: | The proposed framework can be used to estimate the robustness of NLP models. |
Fooling the Textual Fooler via Randomizing Latent Representations (2024.findings-acl)
Copied to clipboard
| Challenge: | Several adversarial attacks can compromise the model without accessing the model architecture or model parameters (i.e., a blackbox setting) Several studies have revealed that deep NLP models are vulnerable to adversarials that slightly perturb the input to cause the models to misbehave. |
| Approach: | They propose a lightweight and attack-agnostic defense that perplexes the process of generating an adversarial example in query-based black-box attacks. |
| Outcome: | The proposed defense is lightweight and attack-agnostic and does not necessitate additional computational overhead during training nor does it rely on assumptions about the potential adversarial perturbation set while having a negligible impact on the model’s accuracy. |
Learning to Discriminate Perturbations for Blocking Adversarial Attacks in Text Classification (D19-1)
Copied to clipboard
| Challenge: | Existing studies on adversarial attacks on deep learning models focus on generation of adversarials and defense against adversarial attacks. |
| Approach: | They propose a framework to identify and adjust malicious perturbations and block adversarial attacks for machine learning models. |
| Outcome: | The proposed framework outperforms baseline methods in blocking adversarial attacks for text classification models. |