| Challenge: | Recent studies indicate that large language models can be meta-prompted to perform automatic prompt engineering, but their potential is limited due to insufficient guidance for complex reasoning in the meta-prompt. |
| Approach: | They propose to infuse three key components into a meta-prompt to guide reasoning . they find prompts that outperform “let’s think step by step” by 6.3% on MultiArith and 3.1% on GSM8K . |
| Outcome: | The proposed method outperforms “let’s think step by step” by 6.3% on MultiArith and 3.1% on GSM8K and outperfies baselines on counterfactual tasks by 6.9%. |
Similar Papers
Prompt Space Optimizing Few-shot Reasoning Success with Large Language Models (2024.findings-naacl)
Copied to clipboard
| Challenge: | Prompt engineering is an essential technique for enhancing the abilities of large language models (LLMs) by providing explicit and specific instructions. |
| Approach: | They propose a new approach that uses text embeddings to obtain basis vectors by matrix decomposition and constructs a space for representing all prompts. |
| Outcome: | The proposed approach significantly outperforms state-of-the-art prompt paradigms on ten public reasoning benchmarks. |
A Systematic Survey of Automatic Prompt Optimization Techniques (2025.emnlp-main)
Copied to clipboard
Kiran Ramnath, Kang Zhou, Sheng Guan, Soumya Smruti Mishra, Xuan Qi, Zhengyuan Shen, Shuai Wang, Sangmin Woo, Sullam Jeoung, Yawei Wang, Haozhu Wang, Han Ding, Yuzhe Lu, Zhichao Xu, Yun Zhou, Balasubramaniam Srinivasan, Qiaojing Yan, Yueyan Chen, Haibo Ding, Panpan Xu, Lin Lee Cheong
| Challenge: | Recent advances in prompt engineering have created impediments for end users to adopt . however, prompt engineering remains an impedance due to rapid advances in models, tasks, and associated best practices. |
| Approach: | They propose to define APO as a 5-part unifying framework and categorize all relevant works based on their salient features. |
| Outcome: | The proposed framework aims to improve the performance of large language models on various tasks. |
SOPL: A Sequential Optimal Learning Approach to Automated Prompt Engineering in Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Using automated prompt engineering to identify effective features is essential for large language models. |
| Approach: | They propose an optimal learning framework for automated prompt engineering for black-box models . feature-based method is used to express prompt templates, which broadens the search space . |
| Outcome: | The proposed learning framework outperforms benchmark strategies on instruction induction tasks with limited budgets. |
Why Prompt Design Matters and Works: A Complexity Analysis of Prompt Search Space in LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Chain-of-Thought (CoT) prompting has emerged as a practical workaround, but most CoT-based methods rely on a single generic prompt like “think step by step” with no task-specific adaptation. |
| Approach: | They propose a theoretical framework that explains why some prompts succeed while others fail by using a generic generic prompt like "think step by step" they show that prompts function as selectors, extracting specific task-relevant information from the model's full hidden state during CoT reasoning. |
| Outcome: | The proposed framework explains why some prompts succeed while others fail. |
promptolution: A Unified, Modular Framework for Prompt Optimization (2026.eacl-demo)
Copied to clipboard
| Challenge: | Existing implementations of prompt optimization are tied to unmaintained, isolated codebases or require invasive integration into application frameworks. |
| Approach: | They propose a unified, modular open-source framework that integrates multiple contemporary discrete prompt optimizers within a single extensible system for both practitioners and researchers. |
| Outcome: | The proposed framework integrates multiple discrete prompt optimizers, supports systematic and reproducible benchmarking, and returns framework-agnostic prompt strings, enabling seamless integration into existing LLM pipelines while remaining agnosite to the underlying model implementation. |
PRompt Optimization in Multi-Step Tasks (PROMST): Integrating Human Feedback and Heuristic-based Sampling (2024.emnlp-main)
Copied to clipboard
| Challenge: | Prompt optimization aims to find the best prompt to a large language model (LLM) for a given task. |
| Approach: | They propose a method to optimize prompts for LLM-driven multi-step tasks using a human-designed feedback rule. |
| Outcome: | The proposed method outperforms human-engineered prompts and several other prompt optimization methods on 11 representative multi-step tasks. |
MAPO: Boosting Large Language Model Performance with Model-Adaptive Prompt Optimization (2023.findings-emnlp)
Copied to clipboard
Yuyan Chen, Zhihao Wen, Ge Fan, Zhengyu Chen, Wei Wu, Dayiheng Liu, Zhixu Li, Bang Liu, Yanghua Xiao
| Challenge: | Existing research emphasizes the importance of adapting prompts to specific tasks, rather than specific LLMs. |
| Approach: | They propose a model-adaptive prompt optimizer method that optimizes original prompts for each LLM in downstream tasks. |
| Outcome: | The proposed method can optimize prompts for an LLM in downstream tasks. |
PromptPrism: A Linguistically-Inspired Taxonomy for Prompts (2026.findings-eacl)
Copied to clipboard
| Challenge: | PromptPrism is a linguistically-inspired taxonomy that enables prompt analysis across three hierarchical levels. |
| Approach: | They propose a linguistically-inspired taxonomy that enables prompt analysis across three hierarchical levels: functional structure, semantic component, and syntactic pattern. |
| Outcome: | The proposed taxonomy bridges traditional language understanding with modern LLM research . it improves prompt quality and improves model performance across tasks . |
PromptWizard: Optimizing Prompts via Task-Aware, Feedback-Driven Self-Evolution (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have transformed AI across diverse domains, with prompting being central to their success in guiding model outputs. |
| Approach: | They propose a framework for discrete prompt optimization that generates human-readable prompts using feedback-driven critique and synthesis process. |
| Outcome: | The proposed framework improves prompt quality across 45 tasks and reduces API calls, token usage and overall cost. |
P3: Prompts Promote Prompting (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in prompt optimization have shown effectiveness of using multiple components to optimize models . however, such unilateral approaches often yield suboptimal results due to interdependent nature of these components. |
| Approach: | They propose a self-improvement framework that optimizes both system and user prompts . they use offline optimized prompts to promote online prompt optimization . |
| Outcome: | The proposed framework improves performance on general and reasoning tasks. |