Jinyu Xiang, Jiayi Zhang, Zhaoyang Yu, Xinbing Liang, Fengwei Teng, Jinhao Tu, Fashen Ren, Xiangru Tang, Sirui Hong, Chenglin Wu, Yuyu Luo
| Challenge: | Existing prompt optimization methods rely heavily on external references such as ground truth or by humans, limiting their applicability in real-world scenarios where such data is unavailable or costly to obtain. |
| Approach: | They propose a cost-efficient framework that discovers effective prompts for both closed and open-ended tasks without external reference. |
| Outcome: | The proposed framework outperforms state-of-the-art prompt optimization methods with significantly lower costs and fewer samples. |
Similar Papers
Local Prompt Optimization (2025.naacl-short)
Copied to clipboard
| Challenge: | Existing prompt optimization methods optimize prompts globally, but they lack the correct words for a task. |
| Approach: | They propose a local prompt optimization process that integrates with any general automatic prompt engineering method to optimize a prompt over a large vocabulary. |
| Outcome: | The proposed method improves on Math Reasoning and BIG-bench Hard benchmarks and shows that it can converge to the optimal prompt faster than global methods. |
PromptWizard: Optimizing Prompts via Task-Aware, Feedback-Driven Self-Evolution (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have transformed AI across diverse domains, with prompting being central to their success in guiding model outputs. |
| Approach: | They propose a framework for discrete prompt optimization that generates human-readable prompts using feedback-driven critique and synthesis process. |
| Outcome: | The proposed framework improves prompt quality across 45 tasks and reduces API calls, token usage and overall cost. |
HIPO: A Hierarchical Prompt Optimization Framework with Task Awareness and Fine-Grained Debugging (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for prompt optimization apply the same prompt across all samples . existing methods ignore variation in sample difficulty . |
| Approach: | They propose a framework that shifts the paradigm from dataset-level to sample-level optimization. |
| Outcome: | The proposed framework outperforms baselines on 27 tasks and reduces API calls, token consumption and overall cost by 1.2 to 80. |
A Systematic Survey of Automatic Prompt Optimization Techniques (2025.emnlp-main)
Copied to clipboard
Kiran Ramnath, Kang Zhou, Sheng Guan, Soumya Smruti Mishra, Xuan Qi, Zhengyuan Shen, Shuai Wang, Sangmin Woo, Sullam Jeoung, Yawei Wang, Haozhu Wang, Han Ding, Yuzhe Lu, Zhichao Xu, Yun Zhou, Balasubramaniam Srinivasan, Qiaojing Yan, Yueyan Chen, Haibo Ding, Panpan Xu, Lin Lee Cheong
| Challenge: | Recent advances in prompt engineering have created impediments for end users to adopt . however, prompt engineering remains an impedance due to rapid advances in models, tasks, and associated best practices. |
| Approach: | They propose to define APO as a 5-part unifying framework and categorize all relevant works based on their salient features. |
| Outcome: | The proposed framework aims to improve the performance of large language models on various tasks. |
LLM Prompt Duel Optimizer: Efficient Label-Free Prompt Optimization (2026.findings-acl)
Copied to clipboard
Yuanchen Wu, Saurabh Verma, Justin Lee, Fangzhou Xiong, Poppy Zhang, Amel Awadelkarim, Xu Chen, Yubai Yuan, Shawndra Hill
| Challenge: | Large language models (LLMs) are highly sensitive to prompts, but most automatic prompt optimization methods assume access to ground-truth references that are costly to obtain. |
| Approach: | They propose a sample-efficient framework for label-free prompt optimization based on pairwise preference feedback from an LLM judge. |
| Outcome: | Experiments on BIG-bench Hard and MS MARCO show that the proposed framework identifies stronger prompts than label-free baselines while offering favorable quality–cost trade-offs. |
SEE: Strategic Exploration and Exploitation for Cohesive In-Context Prompt Optimization (2025.acl-long)
Copied to clipboard
Wendi Cui, Jiaxin Zhang, Zhuohang Li, Hao Sun, Damien Lopez, Kamalika Das, Bradley A. Malin, Sricharan Kumar
| Challenge: | Existing approaches separate the optimization of prompt instructions and in-context learning examples, leading to incohesive, suboptimal results. |
| Approach: | They propose a framework that refines both prompt instructions and in-context learning examples. |
| Outcome: | The proposed framework outperforms state-of-the-art prompt optimization methods on 35 benchmark tasks. |
Rethinking Prompt Optimizers: From Prompt Merits to Optimization (2026.eacl-long)
Copied to clipboard
Zixiao Zhu, Hanzhang Zhou, Zijian Feng, Tianjiao Li, Chua Jia Jim Deryl, Lee Onn Mak, Gee Wah Ng, Kezhi Mao
| Challenge: | Existing methods to optimize prompts rely on LLMs' self-generation ability but lack interpretability due to implicit optimization. |
| Approach: | They propose a model-agnostic prompt quality merits and a merit-guided, locally deployable prompt optimizer trained on a lightweight LLM to improve prompt quality. |
| Outcome: | The proposed model avoids online optimization, reduces privacy concerns, and generalizes effectively to both large-scale and lightweight inference models. |
MAPO: Boosting Large Language Model Performance with Model-Adaptive Prompt Optimization (2023.findings-emnlp)
Copied to clipboard
Yuyan Chen, Zhihao Wen, Ge Fan, Zhengyu Chen, Wei Wu, Dayiheng Liu, Zhixu Li, Bang Liu, Yanghua Xiao
| Challenge: | Existing research emphasizes the importance of adapting prompts to specific tasks, rather than specific LLMs. |
| Approach: | They propose a model-adaptive prompt optimizer method that optimizes original prompts for each LLM in downstream tasks. |
| Outcome: | The proposed method can optimize prompts for an LLM in downstream tasks. |
Self-Renewal Prompt Optimizing with Implicit Reasoning (2024.findings-emnlp)
Copied to clipboard
Zihan Liang, Ben Chen, Zhuoran Ran, Zihan Wang, Huangyu Dai, Yufei Ma, Dehong Gao, Xiaoyan Cai, Libin Yang
| Challenge: | Recent advances in NLP have been driven by the development of Large Language Models (LLMs). |
| Approach: | They propose a self-renewal approach to optimize LLM outputs to better align with human preferences without supervised fine-tuning. |
| Outcome: | The proposed approach improves outputs to better align with human preferences across LLMs and tasks without supervised fine-tuning. |
Enhancing LLM-as-a-Judge through Active-Sampling-based Prompt Optimization (2025.acl-industry)
Copied to clipboard
| Challenge: | Suboptimal prompts can introduce biases, inconsistencies, and unreliable evaluations. |
| Approach: | They propose an active-sampling-based framework for automatic prompt optimization . they use a small, diverse subset of samples to guide prompt refinement . |
| Outcome: | The proposed framework outperforms baselines on four popular LLMs and three real-world datasets. |