Efficient Universal Goal Hijacking with Semantics-guided Prompt Organization (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for prompt injection have focused on optimizing the suffix, overlooking the role of the prompt. |
| Approach: | They propose a method that incorporates an efficient optimization algorithm and two semantics-guided prompt organization strategies to optimize the suffix sequence for universal goal hijacking. |
| Outcome: | The proposed method can generate a fixed suffix that can concatenate to arbitrary user prompts for universal goal hijacking. |
Similar Papers
SEE: Strategic Exploration and Exploitation for Cohesive In-Context Prompt Optimization (2025.acl-long)
Copied to clipboard
Wendi Cui, Jiaxin Zhang, Zhuohang Li, Hao Sun, Damien Lopez, Kamalika Das, Bradley A. Malin, Sricharan Kumar
| Challenge: | Existing approaches separate the optimization of prompt instructions and in-context learning examples, leading to incohesive, suboptimal results. |
| Approach: | They propose a framework that refines both prompt instructions and in-context learning examples. |
| Outcome: | The proposed framework outperforms state-of-the-art prompt optimization methods on 35 benchmark tasks. |
Jailbreaking with Universal Multi-Prompts (2025.findings-naacl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have seen rapid development in recent years, but ethical concerns and new types of attacks have emerged. |
| Approach: | They propose a prompt-based method to jailbreak large language models using universal multi-prompts and an approach for defense that outperforms existing techniques. |
| Outcome: | The proposed method outperforms existing techniques for jailbreaking LLMs using universal multi-prompts. |
Reasoning Hijacking: The Fragility of Reasoning Alignment in Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Current LLM safety research focuses on mitigating **Goal Hijacking**, preventing attackers from redirecting a model’s high-level objective. |
| Approach: | They propose a new adversarial prompt attack paradigm that subverts model judgments by injecting spurious decision criteria without altering the high-level task goal. |
| Outcome: | The proposed model subverts model judgments by injecting spurious decision criteria without altering the high-level task goal. |
Ignore This Title and HackAPrompt: Exposing Systemic Vulnerabilities of LLMs Through a Global Prompt Hacking Competition (2023.emnlp-main)
Copied to clipboard
Sander Schulhoff, Jeremy Pinto, Anaum Khan, Louis-François Bouchard, Chenglei Si, Svetlina Anati, Valen Tagliabue, Anson Kost, Christopher Carnahan, Jordan Boyd-Graber
| Challenge: | Large Language Models are increasingly being deployed in interactive contexts that involve direct user engagement. |
| Approach: | They run a global prompt hacking competition to encourage research on prompt hacks . they elicit 600K+ adversarial prompts against three state-of-the-art LLMs based on a dataset . |
| Outcome: | The results of the competition show that current LLMs can be manipulated via prompt hacking . the competition elicits 600K+ adversarial prompts against three state-of-the-art LLM models . |
Prompterator: Iterate Efficiently towards More Effective Prompts (2023.emnlp-demo)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) use a process known as prompting to solve arbitrary language tasks. prompting is a non-trivial task that requires experimentation in order to arrive at a prompt that solves a specific task. |
| Approach: | They propose a tool that helps users iterate over different potential prompts and choose the best performing one based on human feedback. |
| Outcome: | The proposed tool is open source and easily extensible. |
ProCut: LLM Prompt Compression via Attribution Estimation (2025.emnlp-industry)
Copied to clipboard
| Challenge: | ProCut compresses prompts using attribution analysis to reduce prompt size and latency. |
| Approach: | They propose a framework that compresses prompts through attribution analysis using a heuristic and attribution-based attribution model. |
| Outcome: | The proposed framework reduces prompt size by 78% while maintaining or improving task performance by 62%. |
LLM Prompt Duel Optimizer: Efficient Label-Free Prompt Optimization (2026.findings-acl)
Copied to clipboard
Yuanchen Wu, Saurabh Verma, Justin Lee, Fangzhou Xiong, Poppy Zhang, Amel Awadelkarim, Xu Chen, Yubai Yuan, Shawndra Hill
| Challenge: | Large language models (LLMs) are highly sensitive to prompts, but most automatic prompt optimization methods assume access to ground-truth references that are costly to obtain. |
| Approach: | They propose a sample-efficient framework for label-free prompt optimization based on pairwise preference feedback from an LLM judge. |
| Outcome: | Experiments on BIG-bench Hard and MS MARCO show that the proposed framework identifies stronger prompts than label-free baselines while offering favorable quality–cost trade-offs. |
Prompt Injection in Automated Résumé Screening with Large Language Models: Single and Multi-Injection Settings (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are increasingly used to screen and rank job applicants . a recent study shows that prompt injection can improve ranking when résumé quality is homogeneous . |
| Approach: | They study whether prompt injection distorts quality-based candidate selection . they find that it improves rankings when résumé quality is homogeneous . |
| Outcome: | The proposed model improves rankings when résumé quality is homogeneous and few candidates inject. |
Stop Fixating on Prompts: Reasoning Hijacking and Constraint Tightening for Red-Teaming LLM Agents (2026.acl-long)
Copied to clipboard
| Challenge: | Existing red-team methods rely on modifying user prompts, which lack adaptability to new data and may impact the agent’s performance. |
| Approach: | They propose a framework that implicitly manipulates the agent’s reasoning trajectory and memory retrieval with three key stages: Trigger Extraction, Reasoning Hijacking, and Constraint Tightening. |
| Outcome: | The proposed framework shows outstanding performance in cross-model and cross-scenario environments. |
Defense Against Prompt Injection Attack by Leveraging Attack Techniques (2025.acl-long)
Copied to clipboard
| Challenge: | Recent attacks leverage LLMs’ instruction-following abilities and their inabilities to distinguish instructions injected in the data content. |
| Approach: | They invert the intention of prompt injection methods to develop novel defense methods based on previous training-free attack methods by repeating the attack process with the original input instruction rather than the injected instruction. |
| Outcome: | The proposed methods outperform existing defense approaches, achieving state-of-the-art results. |