Meta-Learning of Prompt Generation for Lightweight Prompt Engineering on Language-Model-as-a-Service

Challenge:	Language-Model-as-a-Services (LMaaSs) support a variety of user tasks through in-context learning from prompts.
Approach:	They propose a lightweight automatic prompt generation method that meta-trains a prompt generation model to enable robust learning from the contexts created by the generated prompts.
Outcome:	The proposed method improves performance on unseen tasks by 19.4% compared to the state-of-the-art prompt generation method.

Similar Papers

Overview

Challenge

Approach

Outcome

The Power of Scale for Parameter-Efficient Prompt Tuning (2021.emnlp-main)

Copied to clipboard

Brian Lester, Rami Al-Rfou, Noah Constant

Responsible & Trustworthy NLP

Challenge:	Unlike discrete text prompts used by GPT-3, soft prompts are learned through backpropagation and can be tuned to incorporate signals from any number of labeled examples.
Approach:	They propose a mechanism for learning "soft prompts" to condition frozen language models to perform specific downstream tasks.
Outcome:	The proposed method outperforms fewshot learning using GPT-3 and matches the quality of model tuning as models exceed billions of parameters.

Learning to Transfer Prompts for Text Generation (2022.naacl-main)

Copied to clipboard

Junyi Li, Tianyi Tang, Jian-Yun Nie, Ji-Rong Wen, Xin Zhao

Language Models

Challenge:	Pretrained language models (PLMs) have made remarkable progress in text generation tasks via fine-tuning.
Approach:	They propose a prompt-based method that learns source prompts and transfers them as target prompts to perform target generation tasks.
Outcome:	The proposed method can be used to perform text generation tasks in a transferable setting.

PromptWizard: Optimizing Prompts via Task-Aware, Feedback-Driven Self-Evolution (2025.findings-acl)

Copied to clipboard

Eshaan Agarwal, Raghav Magazine, Joykirat Singh, Vivek Dani, Tanuja Ganu, Akshay Nambi

Semantic Text Processing

Challenge:	Large language models (LLMs) have transformed AI across diverse domains, with prompting being central to their success in guiding model outputs.
Approach:	They propose a framework for discrete prompt optimization that generates human-readable prompts using feedback-driven critique and synthesis process.
Outcome:	The proposed framework improves prompt quality across 45 tasks and reduces API calls, token usage and overall cost.

Prompt Engineering a Prompt Engineer (2024.findings-acl)

Copied to clipboard

Qinyuan Ye, Mohamed Ahmed, Reid Pryzant, Fereshte Khani

Low-Resource NLP

Challenge:	Recent studies indicate that large language models can be meta-prompted to perform automatic prompt engineering, but their potential is limited due to insufficient guidance for complex reasoning in the meta-prompt.
Approach:	They propose to infuse three key components into a meta-prompt to guide reasoning . they find prompts that outperform “let’s think step by step” by 6.3% on MultiArith and 3.1% on GSM8K .
Outcome:	The proposed method outperforms “let’s think step by step” by 6.3% on MultiArith and 3.1% on GSM8K and outperfies baselines on counterfactual tasks by 6.9%.

P3: Prompts Promote Prompting (2025.findings-acl)

Copied to clipboard

Xinyu Zhang, Yuanquan Hu, Fangchao Liu, Zhicheng Dou

Language Models

Challenge:	Recent advances in prompt optimization have shown effectiveness of using multiple components to optimize models . however, such unilateral approaches often yield suboptimal results due to interdependent nature of these components.
Approach:	They propose a self-improvement framework that optimizes both system and user prompts . they use offline optimized prompts to promote online prompt optimization .
Outcome:	The proposed framework improves performance on general and reasoning tasks.

Learning from Contrastive Prompts: An Automated Prompt Optimization Framework (2026.findings-acl)

Copied to clipboard

Mingqi Li, Karan Aggarwal, Yong Xie, Aitzaz Ahmad, Stephen Lau

Language Models

Challenge:	Existing prompt optimization methods often underperform due to learning exclusively from incorrect samples.
Approach:	They propose a framework that leverages contrastive prompts to distinguish between high- and low-performing cases.
Outcome:	The proposed framework can generalize across open and proprietary models and NLU benchmarks.

Prompting Test-Time Scaling Is A Strong LLM Reasoning Data Augmentation (2026.findings-acl)

Copied to clipboard

Sondos Mahmoud Bsharat, Zhiqiang Shen

Reasoning

Challenge:	Large language models exhibit strong reasoning when guided by chain-of-thought exemplars . collecting large, high-quality reasoning datasets remains laborious and resource-intensive .
Approach:	They propose a prompt-space data augmentation framework for enhancing LLM reasoning . they use a pool of 90 randomly selected reasoning instances to elicit diverse reasoning trajectories .
Outcome:	The proposed framework improves accuracy over small-data benchmarks and generalization on out-of-domain reasoning evaluations.

MutantPrompt: Prompt Optimization via Mutation Under a Budget on Modest-sized LMs (2025.findings-acl)

Copied to clipboard

Arijit Nag, Animesh Mukherjee, Niloy Ganguly, Soumen Chakrabarti

Responsible & Trustworthy NLP

Challenge:	Large Language Models (LLMs) have revolutionized the way we learn and process information, but identifying optimal prompts remains a challenge for low-resource languages.
Approach:	They propose a framework that leverages multi-armed bandit algorithms to efficiently identify optimal prompts tailored to low-resource languages.
Outcome:	The proposed framework is able to find optimal prompts for low-resource languages and significantly improves performance across multiple low-level tasks.

Prompt2Model: Generating Deployable Models from Natural Language Instructions (2023.emnlp-demo)

Copied to clipboard

Vijay Viswanathan, Chenyang Zhao, Amanda Bertsch, Tongshuang Wu, Graham Neubig

Low-Resource NLP

Challenge:	Large language models (LLMs) are a step backward from traditional special-purpose NLP models . they require extensive computational resources for deployment and can be gated behind APIs .
Approach:	They propose a general-purpose method that takes a natural language task description and uses it to train a special-purpose model.
Outcome:	The proposed method outperforms a strong LLM by 20% while being 700 times smaller.

Efficient and Effective Prompt Tuning via Prompt Decomposition and Compressed Outer Product (2025.naacl-long)

Copied to clipboard

Pengxiang Lan, Haoyu Xu, Enneng Yang, Yuliang Liang, Guibing Guo, Jianzhe Zhao, Xingwei Wang

Responsible & Trustworthy NLP

Challenge:	Existing methods for fine-tuning pre-trained language models overlook intrinsic semantic associations between soft prompt tokens, leading to high discreteness and limited interactions.
Approach:	They propose a low-parameters Prompt Tuning method which leverages prompt decomposition and compressed outer product to facilitate multiple interactions among prompt tokens.
Outcome:	Experiments on six architectures and eight datasets show that the proposed method outperforms state-of-the-art methods in performance and efficiency.