| Challenge: | Prompts used in large language model based applications are often fixed and lengthy, leading to significant computational overhead. |
| Approach: | They propose a method that internalizes complex prompts using a joint training approach and a data synthesis technique that auto-collects conversational datasets by swapping roles of agent and environment. |
| Outcome: | The proposed method internalizes complex prompts across agent-based applications and generates the content along with reasons for why it should change accordingly. |
Similar Papers
PromptIntern: Saving Inference Costs by Internalizing Recurrent Prompt during Large Language Model Fine-tuning (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in fine-tuning large language models have greatly enhanced their usage in domain-specific tasks. |
| Approach: | They propose a method which internalizes prompt knowledge during model fine-tuning to achieve efficient inference and save costs. |
| Outcome: | The proposed approach reduces input tokens by 90%, accelerates inference by 4.2 times, and reduces monetary inference costs by 88.3%. |
Prompt-R1: Collaborative Automatic Prompting Framework via End-to-end Reinforcement Learning (2026.findings-acl)
Copied to clipboard
Wenjin Liu, Haoran Luo, Xueyuan Lin, Haoming Liu, Tiesunlong Shen, Jiapu Wang, Rui Mao, Erik Cambria
| Challenge: | Existing large language models are limited in understanding, reasoning, calculation, and generation, limiting their performance in complex reasoning and dynamic tasks. |
| Approach: | They propose a plug-and-play framework that integrates a small-scale LLM (as agent) with large-scale large-level LLMs (a as environment) they propose generating prompts that are used to interact with LLM, and a double constraint reward that optimizes correctness and quality of generation. |
| Outcome: | The proposed framework significantly outperforms baseline large-scale large-language models across various tasks. |
Vector-Quantized Prompt Learning for Paraphrase Generation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for paraphrase generation are difficult to understand and generate. |
| Approach: | They propose to generate diverse paraphrases by using instance-dependent prompts to control the generation of pre-trained models. |
| Outcome: | The proposed method achieves state-of-the-art on three benchmark datasets, including Quora, Wikianswers, and MSCOCO. |
promptolution: A Unified, Modular Framework for Prompt Optimization (2026.eacl-demo)
Copied to clipboard
| Challenge: | Existing implementations of prompt optimization are tied to unmaintained, isolated codebases or require invasive integration into application frameworks. |
| Approach: | They propose a unified, modular open-source framework that integrates multiple contemporary discrete prompt optimizers within a single extensible system for both practitioners and researchers. |
| Outcome: | The proposed framework integrates multiple discrete prompt optimizers, supports systematic and reproducible benchmarking, and returns framework-agnostic prompt strings, enabling seamless integration into existing LLM pipelines while remaining agnosite to the underlying model implementation. |
GenPilot: A Multi-Agent System for Test-Time Prompt Optimization in Image Generation (2025.findings-emnlp)
Copied to clipboard
Wen Ye, Zhaocheng Liu, Gui Yuwei, Tingyu Yuan, Yunyue Su, Bowen Fang, Chaoyang Zhao, Qiang Liu, Liang Wang
| Challenge: | Existing methods for text-to-image synthesis lack systematic error analysis and refinement strategies, resulting in limited reliability and effectiveness. |
| Approach: | They propose a plug-and-play multi-agent system called GenPilot that integrates error analysis, clustering-based adaptive exploration, fine-grained verification and a memory module for iterative optimization. |
| Outcome: | The proposed method improves text consistency and structural coherence on images with a plug-and-play system. |
DLPO: Towards a Robust, Efficient, and Generalizable Prompt Optimization Framework from a Deep-Learning Perspective (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for prompt optimization still face challenges in robustness, efficiency, and generalization. |
| Approach: | They propose 7 new approaches inspired by traditional deep learning paradigms for prompt optimization that integrate text-based gradient optimization. |
| Outcome: | The proposed methods integrate deep learning paradigms into text-based gradient optimization. |
PromptGen: Automatically Generate Prompts using Generative Models (2022.findings-naacl)
Copied to clipboard
| Challenge: | Recent prompt learning has received significant attention, where downstream tasks are reformulated to the mask-filling task with the help of a textual prompt. |
| Approach: | They propose a model PromptGen which can automatically generate prompts conditional on the input sentence. |
| Outcome: | The proposed model outperforms baseline models on the knowledge probing LAMA benchmark. |
PLACES: Prompting Language Models for Social Conversation Synthesis (2023.findings-eacl)
Copied to clipboard
Maximillian Chen, Alexandros Papangelis, Chenyang Tao, Seokhwan Kim, Andy Rosenbaum, Yang Liu, Zhou Yu, Dilek Hakkani-Tur
| Challenge: | Currently, collecting high quality conversational data is expensive and infeasible for many applications . a promising direction is to generate synthetic dialogues by prompting large language models . |
| Approach: | They propose to use expert-written conversations as in-context examples to generate synthetic dialogues by prompting large language models. |
| Outcome: | The proposed approach is generalizable to multi-party conversations, compared to human-collected conversations. |
MetaPrompting: Learning to Learn Better Prompts (2022.coling-1)
Copied to clipboard
| Challenge: | Recent research on prompting moves from discrete tokens based "hard prompts" to continuous "soft prompts", which employ learnable vectors as pseudo prompt tokens and achieve better performance. |
| Approach: | They propose a generalized soft prompting method that uses model-agnostic meta-learning to find better initialization for soft prompts. |
| Outcome: | The proposed method improves on three datasets and brings new state-of-the-art performance. |
IAPT: Instance-Aware Prompt Tuning for Large Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for prompt tuning require many soft tokens to guarantee performance . large language models still require a large amount of GPU memory and computations to fine-tune . |
| Approach: | They propose to use a parameter-efficient soft prompt generator to generate idiosyncratic soft prompts for each input instruction. |
| Outcome: | The proposed method outperforms the baselines with comparable tunable parameters and is more efficient than LoRA under the single-backbone multi-tenant setting. |