Don’t Generate, Classify! Low-Latency Prompt Optimization with Structured Complementary Prompt (2026.eacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated strong performance across diverse tasks, but their performance varies significantly across different prompts. |
| Approach: | They propose a framework that reframes prompt engineering as a classification problem. |
| Outcome: | The proposed framework improves answer quality by up to 26.5% in win rate compared to prior methods while reducing latency by upto 1,956 times. |
Similar Papers
DLPO: Towards a Robust, Efficient, and Generalizable Prompt Optimization Framework from a Deep-Learning Perspective (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for prompt optimization still face challenges in robustness, efficiency, and generalization. |
| Approach: | They propose 7 new approaches inspired by traditional deep learning paradigms for prompt optimization that integrate text-based gradient optimization. |
| Outcome: | The proposed methods integrate deep learning paradigms into text-based gradient optimization. |
FIPO: Free-form Instruction-oriented Prompt Optimization with Preference Dataset and Modular Fine-tuning Schema (2025.coling-main)
Copied to clipboard
| Challenge: | naive prompts can enhance the task performance of large language models, but they are resource-intensive. |
| Approach: | They propose an automatic prompt optimization method that refines naive prompts according to task outputs from in-box testing models. |
| Outcome: | The proposed method is based on a large-scale dataset and performed fairly across multiple models. |
PMPO: Probabilistic Metric Prompt Optimization for Small and Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods evaluate candidate prompts by sampling full outputs, often coupled with self critique or human annotated preferences, which limits scalability, especially for smaller models or models that are not instruction tuned. |
| Approach: | They propose a framework that uses token level cross entropy as a direct, lightweight evaluation signal to evaluate candidate prompts. |
| Outcome: | The proposed framework outperforms prior prompt optimizers across model sizes and datasets. |
Rethinking Prompt Optimizers: From Prompt Merits to Optimization (2026.eacl-long)
Copied to clipboard
Zixiao Zhu, Hanzhang Zhou, Zijian Feng, Tianjiao Li, Chua Jia Jim Deryl, Lee Onn Mak, Gee Wah Ng, Kezhi Mao
| Challenge: | Existing methods to optimize prompts rely on LLMs' self-generation ability but lack interpretability due to implicit optimization. |
| Approach: | They propose a model-agnostic prompt quality merits and a merit-guided, locally deployable prompt optimizer trained on a lightweight LLM to improve prompt quality. |
| Outcome: | The proposed model avoids online optimization, reduces privacy concerns, and generalizes effectively to both large-scale and lightweight inference models. |
SOPL: A Sequential Optimal Learning Approach to Automated Prompt Engineering in Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Using automated prompt engineering to identify effective features is essential for large language models. |
| Approach: | They propose an optimal learning framework for automated prompt engineering for black-box models . feature-based method is used to express prompt templates, which broadens the search space . |
| Outcome: | The proposed learning framework outperforms benchmark strategies on instruction induction tasks with limited budgets. |
Self-Supervised Prompt Optimization (2025.findings-emnlp)
Copied to clipboard
Jinyu Xiang, Jiayi Zhang, Zhaoyang Yu, Xinbing Liang, Fengwei Teng, Jinhao Tu, Fashen Ren, Xiangru Tang, Sirui Hong, Chenglin Wu, Yuyu Luo
| Challenge: | Existing prompt optimization methods rely heavily on external references such as ground truth or by humans, limiting their applicability in real-world scenarios where such data is unavailable or costly to obtain. |
| Approach: | They propose a cost-efficient framework that discovers effective prompts for both closed and open-ended tasks without external reference. |
| Outcome: | The proposed framework outperforms state-of-the-art prompt optimization methods with significantly lower costs and fewer samples. |
Gradient-Guided Multi-Judge Prompt Optimization (2026.acl-long)
Copied to clipboard
ChenZhuo Zhao, Xinda Wang, Pu Zhao, Yue Huang, Junting Lu, Ziqian Liu, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang
| Challenge: | Existing approaches to prompt optimization trade off signal quality against computational cost. |
| Approach: | They propose a framework that uses a first-order gradient approximation to score segment importance in a continuous masking direction. |
| Outcome: | The proposed framework improves efficiency and robustness by using a first-order gradient approximation to score segment importance in a continuous masking direction. |
Fine-Tuning and Prompt Optimization: Two Great Steps that Work Better Together (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent work shows the potential of building more powerful Natural Language Processing systems by composing multiple skills of LMs into pipelines. |
| Approach: | They propose to combine weight and prompt optimization strategies to optimize a modular LM pipeline. |
| Outcome: | The proposed strategies outperform optimizing weights and prompts alone by 60% and 6% on average across LMs and tasks. |
MutantPrompt: Prompt Optimization via Mutation Under a Budget on Modest-sized LMs (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have revolutionized the way we learn and process information, but identifying optimal prompts remains a challenge for low-resource languages. |
| Approach: | They propose a framework that leverages multi-armed bandit algorithms to efficiently identify optimal prompts tailored to low-resource languages. |
| Outcome: | The proposed framework is able to find optimal prompts for low-resource languages and significantly improves performance across multiple low-level tasks. |
Selecting Better Samples from Pre-trained LLMs: A Case Study on Question Generation (2023.findings-acl)
Copied to clipboard
Xingdi Yuan, Tong Wang, Yen-Hsiang Wang, Emery Fine, Rania Abdelghani, Hélène Sauzéon, Pierre-Yves Oudeyer
| Challenge: | Large Language Models (LLMs) have demonstrated impressive prowess in natural language generation. |
| Approach: | They propose a method to select high-quality questions from LLM-generated candidates using round-trip and prompt-based scoring. |
| Outcome: | The proposed approach can select high-quality questions from a set of LLM-generated candidates without modification of the underlying model nor rely on human annotations. |