| Challenge: | Despite advanced capabilities of MLLMs, prompt injection attacks remain vulnerable to emerging security and safety threats. |
| Approach: | They propose a prompt injection attack that manipulates webpage environment to induce web agent to perform attacker-specified action. |
| Outcome: | The proposed attack outperforms baselines on multiple datasets and is highly effective. |
Similar Papers
The Dangers of Indirect Prompt Injection Attacks on LLM-based Autonomous Web Navigation Agents: A Demonstration (2025.emnlp-demos)
Copied to clipboard
| Challenge: | Large Language Model (LLM)-integrated applications are becoming more popular to support, augment, and automate tasks. |
| Approach: | They propose to embed universal adversarial triggers in webpage HTML to hijack agents . they also use a browser-gym agent powered by Llama-3.1 to test their system . |
| Outcome: | The proposed system software is released under the MIT License . |
PIArena: A Platform for Prompt Injection Evaluation (2026.acl-long)
Copied to clipboard
| Challenge: | OWASP identifies prompt injection as the top-1 security risk for large language models (LLMs). |
| Approach: | They propose a unified platform for prompt injection evaluation that integrates state-of-the-art attacks and defenses into a platform. |
| Outcome: | The proposed attack exploits state-of-the-art defenses and generalizes them on diverse datasets and attacks. |
Leave My Images Alone: Preventing Multi-Modal Large Language Models from Analyzing Images via Visual Prompt Injection (2026.acl-long)
Copied to clipboard
| Challenge: | Multimodal large language models are powerful tools for analyzing Internet-scale image data. |
| Approach: | They propose a method to protect images from unauthorized analysis by MLLMs . they embed a perturbation that acts as a visual prompt injection attack on MLMLs if a malicious actor downloads and queries an image . |
| Outcome: | The proposed method protects images from unauthorized analysis by MLLMs . it embeds a perturbation that acts as a visual prompt injection attack on MLMLs if a malicious actor downloads and queries the protected image . |
Defense Against Prompt Injection Attack by Leveraging Attack Techniques (2025.acl-long)
Copied to clipboard
| Challenge: | Recent attacks leverage LLMs’ instruction-following abilities and their inabilities to distinguish instructions injected in the data content. |
| Approach: | They invert the intention of prompt injection methods to develop novel defense methods based on previous training-free attack methods by repeating the attack process with the original input instruction rather than the injected instruction. |
| Outcome: | The proposed methods outperform existing defense approaches, achieving state-of-the-art results. |
Attacking Vision-Language Computer Agents via Pop-ups (2025.acl-long)
Copied to clipboard
| Challenge: | Existing tools for analyzing and testing VLMs are lacking in understanding what types of attacks are possible and what types are effective. |
| Approach: | They propose to integrate pop-ups into existing agent testing environments to attack VLM agents by ignoring them. |
| Outcome: | The proposed attack success rate is 86% and decreases by 47% when integrating pop-ups into existing agent testing environments. |
AdaptAgent: Adapting Multimodal Web Agents with Few-Shot Learning from Human Demonstrations (2025.acl-long)
Copied to clipboard
| Challenge: | State-of-the-art multimodal web agents can perform many web tasks by processing user instructions and interacting with graphical user interfaces (GUIs). |
| Approach: | They propose to build multimodal web agents for few-shot adaptability using human demonstrations to improve their generalization and adaptability. |
| Outcome: | The proposed framework enables both proprietary and open-weights multimodal web agents to adapt to new websites and domains using few human demonstrations. |
Conjunctive Prompt Attacks in Multi-Agent LLM Systems (2026.acl-long)
Copied to clipboard
| Challenge: | Existing defenses do not reliably stop the attack because no single component appears malicious in isolation. |
| Approach: | They study conjunctive prompt attacks where trigger key and adversarial template appear benign alone but activate harmful behavior when routing brings them together. |
| Outcome: | The proposed model significantly improves performance over baselines while keeping false activations low. |
PromptFix: Few-shot Backdoor Removal via Adversarial Prompt Tuning (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing studies have shown that pre-trained language models can be backdoored such that model behavior is manipulated when trigger tokens are presented. |
| Approach: | They propose a backdoor mitigation strategy for NLP models via adversarial prompt-tuning in few-shot settings that uses two extra sets of soft tokens which approximate the trigger and counteract it respectively. |
| Outcome: | The proposed method keeps model parameters intact and only utilizes two extra sets of soft tokens which approximate the trigger and counteract it respectively. |
Rethinking Assessments of Prompt Injection Attacks (2026.findings-acl)
Copied to clipboard
| Challenge: | Prompt injection attacks are recognized as one of the primary risks faced by LLM-integrated applications in recent years. |
| Approach: | They evaluate prompt injection attacks on LLM-integrated applications across 37 target tasks, 185 injected tasks, 21 attack instructions, and 143,745 queries. |
| Outcome: | The proposed framework provides a solid foundation for assessing vulnerabilities in LLM-integrated applications and evaluating the efficacy of defensive strategies. |
TopicAttack: An Indirect Prompt Injection Attack via Topic Transition (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent attacks have demonstrated potential, but their abrupt instruction injection often undermines their effectiveness. |
| Approach: | They propose a method that prompts the LLM to generate a fabricated conversational transition prompt that gradually shifts the topic toward the injected instruction. |
| Outcome: | The proposed method achieves state-of-the-art performance with an attack success rate (ASR) over 90% in most cases, even when various defense methods are applied. |