Maximum Entropy Loss, the Silver Bullet Targeting Backdoor Attacks in Pre-trained Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing backdoor defense paradigms focus on detecting and removing poisoned samples at pre-training or inference time. |
| Approach: | They propose a new approach where the backdoor attack is directly reversed by incorporating maximum entropy loss into training to neutralize the minimal cross-entropiness loss fine-tuning on poisoned data. |
| Outcome: | The proposed model significantly lowers the attack success rate on classification tasks and reduces the risk of backdoor attacks on clean data. |
Similar Papers
PKAD: Pretrained Knowledge is All You Need to Detect and Mitigate Textual Backdoor Attacks (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Current defense methods can be classified into inference-time and training-time ones based on their execution phase. |
| Approach: | They propose a two-stage poison detection strategy using pre-trained language models to detect poisoned samples before model training. |
| Outcome: | The proposed method achieves better performance than current methods more quickly and with fewer training costs. |
UOR: Universal Backdoor Attacks on Pre-trained Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to attack pre-trained language models rely on manual selection of triggers and backdoor representations. |
| Approach: | They propose a backdoor attack method that turns manual selection into automatic optimization . they propose to use poisoned contrastive learning to learn more uniform backdoor representations . |
| Outcome: | The proposed method achieves better attack performance on text classification tasks compared to manual methods. |
Fine-mixing: Mitigating Backdoors in Fine-tuned Language Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for defending NLP models against backdoors have ignored the clean weights of PLMs. |
| Approach: | They exploit pre-trained weights to mitigate backdoors in fine-tuned NLP models . they use a fine-mixing technique and an Embedding Purification technique to do the same . |
| Outcome: | The proposed method outperforms baseline mitigation methods on three single-sentence sentiment classification tasks and two sentence-pair classification tasks. |
Textual Backdoor Attacks Can Be More Harmful via Two Simple Tricks (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing textual backdoor attacks are vulnerable to backdoors . researchers add extra training task to distinguish poisoned and clean data . |
| Approach: | They propose two tricks that make existing backdoor attacks much more harmful . first trick is to add an extra task to distinguish poisoned and clean data . second trick is using all the clean training data rather than the original clean data. |
| Outcome: | The proposed tricks can significantly improve attack performance in three tough situations including clean data fine-tuning, low-poisoning-rate, and label-consistent attacks. |
Universal Vulnerabilities in Large Language Models: Backdoor Attacks for In-context Learning (2024.emnlp-main)
Copied to clipboard
| Challenge: | In-context learning has shown high efficacy in several NLP tasks, especially in few-shot settings. |
| Approach: | They propose a backdoor attack method that poisons demonstration examples and poisons the demonstration context, preserving the model's generality. |
| Outcome: | The proposed method can make models behave in alignment with predefined intentions without fine-tuning the model. |
Forget the Unneeded: Backdooring Large Language Models via Contrastive-enhanced Machine Unlearning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for prompt tuning for Large Language Models find backdoor attacks to be significant in data-rich scenarios. |
| Approach: | They propose a backdoor attacks through contrastive-enhanced machine unlearning in data-limited scenarios . they use a machine un learning method to capture precise backdoor patterns . |
| Outcome: | The proposed method captures precise backdoor patterns without association between triggers and backdoors, reducing side effects. |
SynGhost: Invisible and Universal Task-agnostic Backdoor Attack via Syntactic Transfer (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing attacks are classified into end-to-end and pre-training types based on the attack phase . Existing backdoor attacks are based upon perplexity, fine-pruning, and maxEntropy. |
| Approach: | They propose an entropy-based poisoning filter that mitigates backdoor attacks . they propose an invisible and universal task-agnostic backdoor attack via syntactic transfer . |
| Outcome: | The proposed attack can transfer backdoors to various downstream tasks while preserving pre-trained language models' pre-training capabilities. |
Backdoor Attacks on Pre-trained Models by Layerwise Weight Poisoning (2021.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained models can be maliciously poisoned with certain triggers, causing a security threat. |
| Approach: | They propose a stronger weight poisoning attack method that introduces a layerwise weight poison strategy to plant deeper backdoors. |
| Outcome: | The proposed method can be widely applied and provide hints for future models robustness studies. |
Large Language Models Are Better Adversaries: Exploring Generative Clean-Label Backdoor Attacks Against Text Classifiers (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Backdoor attacks manipulate model predictions by inserting malicious "poison" instances that contain a specific pattern or "trigger." |
| Approach: | They propose an attack that inserts style-based triggers into training and test data by using a poison selection technique to improve the effectiveness of both LLMBkd and existing backdoor attacks. |
| Outcome: | The proposed attack achieves high success rates across a wide range of styles with little effort and no model training. |
Defending Against Weight-Poisoning Backdoor Attacks for Parameter-Efficient Fine-Tuning (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for parameter-efficient fine-tuning (PEFT) are not effective for weight-poisoning backdoor attacks. |
| Approach: | They propose a parameter-efficient fine-tuning (PEFT) method that updates only a limited set of model parameters and provides a robust defense against weight-poisoning backdoor attacks. |
| Outcome: | The proposed method identifies poisoned samples through confidence and is robust against weight-poisoning backdoor attacks. |