Obliviate: Neutralizing Task-agnostic Backdoors within the Parameter-efficient Fine-tuning Paradigm (2025.findings-naacl)
Copied to clipboard
| Challenge: | Parameter-efficient fine-tuning (PEFT) is a key training strategy for large language models, but its reliance on fewer trainable parameters poses security risks, such as task-agnostic backdoors. |
| Approach: | They propose a method that amplifies benign neurons within PEFT layers and penalizes the influence of trigger tokens. |
| Outcome: | The proposed method reduces the success rate of state-of-the-art task-agnostic backdoors by 83.6%. |
Similar Papers
Defending Against Weight-Poisoning Backdoor Attacks for Parameter-Efficient Fine-Tuning (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for parameter-efficient fine-tuning (PEFT) are not effective for weight-poisoning backdoor attacks. |
| Approach: | They propose a parameter-efficient fine-tuning (PEFT) method that updates only a limited set of model parameters and provides a robust defense against weight-poisoning backdoor attacks. |
| Outcome: | The proposed method identifies poisoned samples through confidence and is robust against weight-poisoning backdoor attacks. |
ELBA-Bench: An Efficient Learning Backdoor Attacks Benchmark for Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing backdoor models are limited in coverage of attack, system integrity and backdoor alignment . ELBA-Bench provides over 1300 experiments encompassing 12 attack methods, 18 datasets, and 12 LLMs. |
| Approach: | They propose a framework that allows attackers to inject backdoor through parameter efficient fine-tuning or without fine-uning techniques. |
| Outcome: | ELBA-Bench provides over 1300 experiments encompassing 12 attack methods, 18 datasets, and 12 LLMs. |
A Gradient Control Method for Backdoor Attacks on Parameter-Efficient Tuning (2023.acl-long)
Copied to clipboard
| Challenge: | Parameter-Efficient Tuning (PET) fine-tunes pre-trained language models for downstream tasks, but a large reduction in the number of attackable parameters will greatly affect the effectiveness of backdoor attacks, resulting in backdoor forgetting. |
| Approach: | They propose a gradient control method to consolidate the attack effect by freezing most parameters of the pre-trained model and fine-tuning only a small number of parameters. |
| Outcome: | The proposed method improves sentiment classification and spam detection, and can be applied to different tasks. |
DiSec: Mitigating Backdoors in Pre-trained Language Models via Disentanglement of Adversarial Weights for Secure Fine-Tuning (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing defenses rely on privileged assumptions, limiting their applicability in realistic settings. |
| Approach: | They propose a task-agnostic backdoor attack that contaminates pre-trained language models . authors propose auxiliary text purification framework that uses only clean auxiliary data . |
| Outcome: | The proposed framework suppresses attack success while preserving clean-task utility. |
Multi-target Backdoor Attacks for Code Pre-trained Models (2023.acl-long)
Copied to clipboard
| Challenge: | Existing work for backdoor attacks on neural code models insert triggers into task-specific data for code-related downstream tasks, limiting the scope of attacks. |
| Approach: | They propose task-agnostic backdoor attacks for code pre-trained models . they use two learning strategies to implant backdoors into code understanding and generation models - Poisoned Seq2Seq learning and token representation learning . |
| Outcome: | The proposed model is pre-trained with two learning strategies to support the multi-target attack of downstream code understanding and generation tasks. |
PAFT: Prompt-Agnostic Fine-Tuning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Prompt-agnostic fine-tuning (PAFT) improves performance by reducing overfitting to specific prompts. |
| Approach: | They propose a method that enhances robustness through dynamic prompt variation during training. |
| Outcome: | The proposed method achieves higher generalization accuracy on unseen prompts than standard methods with similar training efficiency. |
Task-Agnostic Detector for Insertion-Based Backdoor Attacks (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for textual backdoor detection are task-specific and less effective beyond sentence classification. |
| Approach: | They propose a task-agnostic method for backdoor detection that leverages final layer logits and an efficient pooling technique. |
| Outcome: | TABDet can jointly learn from diverse task-specific models, demonstrating superior detection efficacy over traditional methods. |
Look Within or Beyond? A Theoretical Comparison Between Parameter-Efficient and Full Fine-Tuning (2026.acl-long)
Copied to clipboard
| Challenge: | Parameter-Efficient Fine-Tuning (PEFT) is an alternative to Full-Parameter Fine-tuning, but its effectiveness on complex tasks such as reasoning and instruction-following remains unclear. |
| Approach: | They propose to use PEFT to reduce the number of trainable parameters while freezing the weights of LLMs. |
| Outcome: | The proposed methods perform well on standard tasks, but weaknesses on complex and adversarial settings call for new directions beyond current paradigms. |
Jailbreak-Tuning: Models Efficiently Learn Jailbreak Susceptibility (2025.emnlp-main)
Copied to clipboard
Brendan Murphy, Dillon Bowen, Shahrad Mohammadzadeh, Tom Tseng, Julius Broomfield, Adam Gleave, Kellin Pelrine
| Challenge: | a recent study shows that fine-tuning can produce helpful-only models with safeguards destroyed. |
| Approach: | They propose a method for fine-tuning models to generate detailed, high-quality responses to harmful requests. |
| Outcome: | The proposed method produces helpful-only models with safeguards destroyed . OpenAI, Google, and Anthropic models will fully comply with requests for CBRN assistance . |
Parameter-Efficient Fine-Tuning without Introducing New Latency (2023.acl-long)
Copied to clipboard
| Challenge: | Parameter-efficient fine-tuning of pre-trained language models has been demonstrated to be effective, but its inherent characteristics limit its performance. |
| Approach: | They propose to generate a sparse mask in a task-agnostic manner by modifying only a small subset of existing parameters and adding new parameters. |
| Outcome: | The proposed method surpasses existing methods on the GLUE benchmark by a significant margin. |