Efficient and Accurate Prompt Optimization: the Benefit of Memory in Exemplar-Guided Reflection (2025.acl-long)
Copied to clipboard
Cilin Yan, Jingyun Wang, Lin Zhang, Ruihui Zhao, Xiaopu Wu, Kai Xiong, Qingsong Liu, Guoliang Kang, Yangyang Kang
| Challenge: | Recent work utilizes feedbacks generated from erroneous cases to guide prompt optimization . previous methods rely on computational resources and powerful GPUs . |
| Approach: | They propose an automatic prompt engineering method that leverages feedbacks from erroneous cases to guide prompt optimization. |
| Outcome: | The proposed method surpasses state-of-the-art methods with less steps and lower computational resources. |
Similar Papers
Self-Supervised Prompt Optimization (2025.findings-emnlp)
Copied to clipboard
Jinyu Xiang, Jiayi Zhang, Zhaoyang Yu, Xinbing Liang, Fengwei Teng, Jinhao Tu, Fashen Ren, Xiangru Tang, Sirui Hong, Chenglin Wu, Yuyu Luo
| Challenge: | Existing prompt optimization methods rely heavily on external references such as ground truth or by humans, limiting their applicability in real-world scenarios where such data is unavailable or costly to obtain. |
| Approach: | They propose a cost-efficient framework that discovers effective prompts for both closed and open-ended tasks without external reference. |
| Outcome: | The proposed framework outperforms state-of-the-art prompt optimization methods with significantly lower costs and fewer samples. |
Data-Efficient Automatic Prompt Optimization for Memory-Enhanced Conversational Agents (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Automatic prompt optimization (APO) uses algorithms to optimize prompts for LLMs . but application to memory-enhanced conversational agents presents unique challenges . |
| Approach: | They propose a framework for automatic prompt optimization for memory-enhanced conversational agents . they leverage LLMs to holistically optimize the prompts of all agents based on memory writing, reading, and response generation . |
| Outcome: | The proposed framework is applied to memory-enhanced conversational agents . it provides a holistic quality score for responses and performs error attribution . |
Rethinking Prompt Optimizers: From Prompt Merits to Optimization (2026.eacl-long)
Copied to clipboard
Zixiao Zhu, Hanzhang Zhou, Zijian Feng, Tianjiao Li, Chua Jia Jim Deryl, Lee Onn Mak, Gee Wah Ng, Kezhi Mao
| Challenge: | Existing methods to optimize prompts rely on LLMs' self-generation ability but lack interpretability due to implicit optimization. |
| Approach: | They propose a model-agnostic prompt quality merits and a merit-guided, locally deployable prompt optimizer trained on a lightweight LLM to improve prompt quality. |
| Outcome: | The proposed model avoids online optimization, reduces privacy concerns, and generalizes effectively to both large-scale and lightweight inference models. |
FastMem: Fast Memorization of Prompt Improves Context Awareness of Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models struggle with context awareness, leading to inaccuracies in tasks requiring faithful adherence to provided information. |
| Approach: | They propose a method to enhance LLMs' context awareness by updating only the last Feed-Forward Network module to maximize the likelihood of the prompt before inference . |
| Outcome: | The proposed method improves the accuracy of Llama 3-8B-Inst on the NQ-SWAP dataset from 59.1% to 71.6% and reduces the output structure failure rate of Qwen 1.5-4B-Chat from 34.9% to 25.5%. |
ADO: Automatic Data Optimization for Inputs in LLM Prompts (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent research has focused on refining instruction components and augmenting input data with in-context examples, but this study explores the potential benefits of optimizing the input data itself. |
| Approach: | They propose a content engineering and structural reformulation strategy to optimize input data within prompts to improve performance of Large Language Models. |
| Outcome: | The proposed approach improves performance of Large Language Models (LLMs) in various tasks, offering a promising avenue for future research in prompt engineering. |
A Systematic Survey of Automatic Prompt Optimization Techniques (2025.emnlp-main)
Copied to clipboard
Kiran Ramnath, Kang Zhou, Sheng Guan, Soumya Smruti Mishra, Xuan Qi, Zhengyuan Shen, Shuai Wang, Sangmin Woo, Sullam Jeoung, Yawei Wang, Haozhu Wang, Han Ding, Yuzhe Lu, Zhichao Xu, Yun Zhou, Balasubramaniam Srinivasan, Qiaojing Yan, Yueyan Chen, Haibo Ding, Panpan Xu, Lin Lee Cheong
| Challenge: | Recent advances in prompt engineering have created impediments for end users to adopt . however, prompt engineering remains an impedance due to rapid advances in models, tasks, and associated best practices. |
| Approach: | They propose to define APO as a 5-part unifying framework and categorize all relevant works based on their salient features. |
| Outcome: | The proposed framework aims to improve the performance of large language models on various tasks. |
SEE: Strategic Exploration and Exploitation for Cohesive In-Context Prompt Optimization (2025.acl-long)
Copied to clipboard
Wendi Cui, Jiaxin Zhang, Zhuohang Li, Hao Sun, Damien Lopez, Kamalika Das, Bradley A. Malin, Sricharan Kumar
| Challenge: | Existing approaches separate the optimization of prompt instructions and in-context learning examples, leading to incohesive, suboptimal results. |
| Approach: | They propose a framework that refines both prompt instructions and in-context learning examples. |
| Outcome: | The proposed framework outperforms state-of-the-art prompt optimization methods on 35 benchmark tasks. |
PRompt Optimization in Multi-Step Tasks (PROMST): Integrating Human Feedback and Heuristic-based Sampling (2024.emnlp-main)
Copied to clipboard
| Challenge: | Prompt optimization aims to find the best prompt to a large language model (LLM) for a given task. |
| Approach: | They propose a method to optimize prompts for LLM-driven multi-step tasks using a human-designed feedback rule. |
| Outcome: | The proposed method outperforms human-engineered prompts and several other prompt optimization methods on 11 representative multi-step tasks. |
Prompterator: Iterate Efficiently towards More Effective Prompts (2023.emnlp-demo)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) use a process known as prompting to solve arbitrary language tasks. prompting is a non-trivial task that requires experimentation in order to arrive at a prompt that solves a specific task. |
| Approach: | They propose a tool that helps users iterate over different potential prompts and choose the best performing one based on human feedback. |
| Outcome: | The proposed tool is open source and easily extensible. |
Model Performance-Guided Evaluation Data Selection for Effective Prompt Optimization (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing prompt engineering methods rely on randomly selected evaluation subsets, leading to suboptimal prompts. |
| Approach: | They propose an iterative evaluation data selection approach for effective prompt optimization using real time model performance. |
| Outcome: | The proposed approach improves effectiveness by 1.6% to 3.1% and stability by 50% to 55.5% on two datasets BIG-bench and LIAR and two models GPT-3.5 and GPT-4o-mini. |