| Challenge: | Dense retrievers have been shown to be vulnerable to corpus poisoning attacks . a malicious user injects a small fraction of adversarial passages into the retrieval corpus to trick the system into returning these passages among the top-ranked results for a broad set of user queries. |
| Approach: | They propose an attack on dense retrieval systems based on the widely used HotFlip method for efficiently generating adversarial passages. |
| Outcome: | The proposed method achieves a high success rate on several datasets and using several retrievers, and can generalize to unseen queries and new domains. |
Similar Papers
Poisoning Retrieval Corpora by Injecting Adversarial Passages (2023.emnlp-main)
Copied to clipboard
| Challenge: | Dense retrievers have outperformed traditional lexical methods in a range of information retrieval tasks, but to what extent can they be safely deployed in real-world applications? |
| Approach: | They propose a method where a malicious user injects a small number of adversarial passages into a retrieval corpus to maximize similarity with a set of training queries. |
| Outcome: | The proposed attack fools retrieval systems into returning top results for queries not seen by the attacker. |
Tricking Retrievers with Influential Tokens: An Efficient Black-Box Corpus Poisoning Attack (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for crafting adversarial passages are slow and computationally expensive, requiring either access to retriever’s gradients or large computational resources. |
| Approach: | They propose a method that leverages two key characteristics of retrievers: insensitivity to token order and bias towards influential tokens to generate effective adversarial passages. |
| Outcome: | The proposed method achieves superior efficiency and scalability compared to existing methods while maintaining comparable or better attack success rates across multiple datasets. |
Safeguarding RAG Pipelines with GMTP: A Gradient-based Masked Token Probability Method for Poisoned Document Detection (2025.findings-acl)
Copied to clipboard
| Challenge: | Retrieval-Augmented Generation (RAG) provides external knowledge for accurate and up-to-date responses, but external knowledge is vulnerable to poisoning and unauthorized injections. |
| Approach: | They propose a Gradient-based Masked Token Probability defense method to detect and filter out adversarially crafted documents by examining gradients of the retriever’s similarity function. |
| Outcome: | Experiments show that the proposed method eliminates over 90% of poisoned content while retaining relevant documents. |
One Shot Dominance: Knowledge Poisoning Attack on Retrieval-Augmented Generation Systems (2025.findings-emnlp)
Copied to clipboard
Zhiyuan Chang, Mingyang Li, Xiaojun Jia, Junjie Wang, Yuekai Huang, Ziyou Jiang, Yang Liu, Qing Wang
| Challenge: | Existing knowledge poisoning attacks against RAG systems require multiple poisoned documents or can only function effectively on simplistic queries. |
| Approach: | They propose a more realistic knowledge poisoning attack that poisons only a single document while remaining effective for complex multi-hop questions involving complex relationships between multiple elements. |
| Outcome: | The proposed attack achieves success by poisoning only a single document while remaining effective for complex multi-hop questions involving complex relationships between multiple elements. |
AIP: Subverting Retrieval-Augmented Generation via Adversarial Instructional Prompt (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing RAG attacks rely on manipulating user queries, but exploit instructional prompts to manipulate RAG outputs covertly. |
| Approach: | They propose an attack that exploits adversarial instructional prompts to manipulate RAG outputs . they propose a query generation strategy that simulates realistic linguistic variation in user queries . |
| Outcome: | The proposed attack exploits instructional prompts to manipulate RAG outputs . it achieves up to 95.23% attack success rate while maintaining benign functionality . |
Token-Level Precise Attack on RAG: Searching for the Best Alternatives to Mislead Generation (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing approaches to attack large language models rely heavily on retrieval and generation stages, limiting their effectiveness in black-box scenarios. |
| Approach: | They propose a retrieval-augmented generation framework that leverages a white-box LLM as an attacker to generate and iteratively optimize malicious passages at the token level. |
| Outcome: | The proposed framework outperforms existing approaches in retrieval-stage and end-to-end attacks on black-box RAG systems. |
Retrievals Can Be Detrimental: Unveiling the Backdoor Vulnerability of Retrieval-Augmented Diffusion Models (2026.acl-long)
Copied to clipboard
| Challenge: | Retrieval-augmented diffusion models (RDMs) have been developed to enhance performance with reduced parameters. |
| Approach: | They propose to integrate retrieval-augmented diffusion models with Retrieval-augmented generation (RAG) that enhances performance with reduced parameters. |
| Outcome: | The proposed framework achieves outstanding attack effects while maintaining benign utility. |
EcoSafeRAG: Efficient Security through Context Analysis in Retrieval-Augmented Generation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing defense methods rely on internal knowledge of the model, which conflicts with the design concept of Retrieval-Augmented Generation (RAG). |
| Approach: | EcoSafeRAG uses sentence-level processing and bait-guided context diversity detection to identify malicious content . |
| Outcome: | EcoSafeRAG uses sentence-level processing and bait-guided context diversity detection to identify malicious content. |
Collapse of Dense Retrievers: Short, Early, and Literal Biases Outranking Factual Evidence (2025.acl-long)
Copied to clipboard
| Challenge: | Notably, when multiple biases combine, models exhibit catastrophic performance degradation, selecting the answer-containing document in less than 10% of cases over a synthetic biased document without the answer. |
| Approach: | They repurpose a relation extraction dataset to quantify the impact of heuristic biases on retrievers like Dragon+ and Contriever. |
| Outcome: | The proposed models exhibit catastrophic performance degradation when multiple biases combine, selecting the answer-containing document in less than 10% of cases over a synthetic biased document without the answer. |
HotFlip: White-Box Adversarial Examples for Text Classification (P18-2)
Copied to clipboard
| Challenge: | Existing methods to create adversarial examples without explicit knowledge of model parameters are not effective. |
| Approach: | They propose an efficient method to generate white-box adversarial examples to trick a character-level neural classifier by an atomic flip operation. |
| Outcome: | The proposed method can be adapted to attack a word-level classifier with a few constraints. |