Are All Prompt Components Value-Neutral? Understanding the Heterogeneous Adversarial Robustness of Dissected Prompt in LLMs

Challenge:	Existing studies treat prompts as flat text, overlooking their internal structure, and different components within a prompt contribute unequally to robustness.
Approach:	They propose a framework that decomposes prompts into functional components and a method that selectively modifies components to expose component-wise vulnerabilities.
Outcome:	The proposed framework exposes component-wise vulnerabilities while ensuring linguistic plausibility through perplexity-based filtering.

Similar Papers

Overview

Challenge

Approach

Outcome

Beyond Prompt Engineering: A Systematic Analysis of Prompt Lexical Sensitivity and Its Impacts on Quality (2026.findings-acl)

Copied to clipboard

Qipeng Xie, Zi Liang, Jiafei Wu, Yufei Chen, Weizheng Wang, Wenao Ma, Zhong Ming, Haiqin Yang, Kaishun Wu

Responsible & Trustworthy NLP

Challenge:	Existing studies on prompt engineering have focused on optimizing models for performance under stylistic perturbations.
Approach:	They conduct the first analysis of n-gram token-level mechanisms . they find that higher average performance is inherently associated with lower variance and greater stability.
Outcome:	The proposed model reduces the variance of the generated code by 40% . the proposed model is based on a large-scale dataset of 132,000 prompt variants .

PromptPrism: A Linguistically-Inspired Taxonomy for Prompts (2026.findings-eacl)

Copied to clipboard

Sullam Jeoung, Yueyan Chen, Yi Zhang, Shuai Wang, Haibo Ding, Lin Lee Cheong

Responsible & Trustworthy NLP

Challenge:	PromptPrism is a linguistically-inspired taxonomy that enables prompt analysis across three hierarchical levels.
Approach:	They propose a linguistically-inspired taxonomy that enables prompt analysis across three hierarchical levels: functional structure, semantic component, and syntactic pattern.
Outcome:	The proposed taxonomy bridges traditional language understanding with modern LLM research . it improves prompt quality and improves model performance across tasks .

Ignore This Title and HackAPrompt: Exposing Systemic Vulnerabilities of LLMs Through a Global Prompt Hacking Competition (2023.emnlp-main)

Copied to clipboard

Sander Schulhoff, Jeremy Pinto, Anaum Khan, Louis-François Bouchard, Chenglei Si, Svetlina Anati, Valen Tagliabue, Anson Kost, Christopher Carnahan, Jordan Boyd-Graber

Responsible & Trustworthy NLP

Challenge:	Large Language Models are increasingly being deployed in interactive contexts that involve direct user engagement.
Approach:	They run a global prompt hacking competition to encourage research on prompt hacks . they elicit 600K+ adversarial prompts against three state-of-the-art LLMs based on a dataset .
Outcome:	The results of the competition show that current LLMs can be manipulated via prompt hacking . the competition elicits 600K+ adversarial prompts against three state-of-the-art LLM models .

promptolution: A Unified, Modular Framework for Prompt Optimization (2026.eacl-demo)

Copied to clipboard

Tom Zehle, Timo Heiß, Moritz Schlager, Matthias Aßenmacher, Matthias Feurer

Semantic Text Processing

Challenge:	Existing implementations of prompt optimization are tied to unmaintained, isolated codebases or require invasive integration into application frameworks.
Approach:	They propose a unified, modular open-source framework that integrates multiple contemporary discrete prompt optimizers within a single extensible system for both practitioners and researchers.
Outcome:	The proposed framework integrates multiple discrete prompt optimizers, supports systematic and reproducible benchmarking, and returns framework-agnostic prompt strings, enabling seamless integration into existing LLM pipelines while remaining agnosite to the underlying model implementation.

ProSA: Assessing and Understanding the Prompt Sensitivity of LLMs (2024.findings-emnlp)

Copied to clipboard

Jingming Zhuo, Songyang Zhang, Xinyu Fang, Haodong Duan, Dahua Lin, Kai Chen

Responsible & Trustworthy NLP

Challenge:	Recent research has neglected instances-level prompt variations and their implications on subjective evaluations.
Approach:	They propose a framework to evaluate and comprehend prompt sensitivity in large language models.
Outcome:	The proposed framework evaluates and comprehends prompt sensitivity in large language models.

The Death and Life of Great Prompts: Analyzing the Evolution of LLM Prompts from the Structural Perspective (2024.emnlp-main)

Copied to clipboard

Yihan Ma, Xinyue Shen, Yixin Wu, Boyang Zhang, Michael Backes, Yang Zhang

Semantic Text Processing

Challenge:	Recent research has shown that high-quality prompts are essential for LLMs to produce accurate and relevant responses.
Approach:	They analyze 10,538 in-the-wild prompts collected from various platforms and develop a framework that decomposes the prompts into eight key components.
Outcome:	The proposed framework decomposes 10,538 in-the-wild prompts into eight components.

Rethinking Prompt-based Debiasing in Large Language Model (2025.findings-acl)

Copied to clipboard

Xinyi Yang, Runzhe Zhan, Shu Yang, Junchao Wu, Lidia S. Chao, Derek F. Wong

Responsible & Trustworthy NLP

Challenge:	Existing prompt-based methods for debiasing are often superficial and lack a thorough understanding of complex bias concepts.
Approach:	They analyze a BBQ and stereoSet benchmarks to examine the assumption that large language models understand biases.
Outcome:	The proposed model misclassified 90% of unbiased content as biased despite high accuracy on BBQ dataset . the proposed model may have been flawed in previous attempts to debiase .

Are My Optimized Prompts Compromised? Exploring Vulnerabilities of LLM-based Optimizers (2026.eacl-long)

Copied to clipboard

Andrew Zhao, Reshmi Ghosh, Vitor R. Carvalho, Emily Lawton, Keegan Hines, Gao Huang, Jack W. Stokes

Responsible & Trustworthy NLP

Challenge:	Recent studies have focused on poisoning during supervised fine-tuning, RLHF, or inference-time time optimization.
Approach:	They propose a simple fake reward attack that requires no access to the reward model and significantly increases vulnerability.
Outcome:	The proposed attack reduces the fake reward ASR from 0.23 to 0.07 without degrading utility.

PromptKeeper: Safeguarding System Prompts for LLMs (2025.findings-emnlp)

Copied to clipboard

Zhifeng Jiang, Zhihua Jin, Guoliang He

Responsible & Trustworthy NLP

Challenge:	PromptKeeper is a defense mechanism designed to safeguard system prompts . adversarial and regular queries can exploit LLM vulnerabilities to expose hidden prompts.
Approach:	PromptKeeper is a defense mechanism designed to safeguard system prompts . it detects both explicit and subtle leakage and regenerates responses using a dummy prompt .
Outcome:	PromptKeeper detects and mitigates side-channel vulnerabilities when prompts are exposed . it regenerates responses using a dummy prompt, ensuring outputs remain indistinguishable from typical interactions .

The Threat of PROMPTS in Large Language Models: A System and User Prompt Perspective (2025.findings-acl)

Copied to clipboard

Zixuan Xia, Haifeng Sun, Jingyu Wang, Qi Qi, Huazheng Wang, Xiaoyuan Fu, Jianxin Liao

Responsible & Trustworthy NLP

Challenge:	Prompts are essential for guiding model output and influencing content generation.
Approach:	They propose to attack models with prompt leakage and prompt jailbreak attacks . they summarize the experimental setups of these methods and explore the relationship between prompt threats and prompt injection attacks.
Outcome:	The proposed methods summarize the experimental setups and examine the relationship between prompt threats and prompt injection attacks.