CRL-Prompt: Contrastive and Reinforcement Learning for Soft Prompt Tuning for Text Classification (2026.acl-srw)
Copied to clipboard
| Challenge: | Manual prompt engineering is time-consuming, non-scalable, and brittle, while current auto-prompting techniques are far from maturity. |
| Approach: | They propose a two-stage method for prompt learning of frozen language models, CRL-Prompt, based on soft prompt initialization followed by contrastive and reinforcement-based refinement. |
| Outcome: | The proposed method achieves consistent improvements over baseline prompt tuning strategies, with gains of up to 2.2% while training fewer than 0.25% of model parameters. |
Similar Papers
The Power of Scale for Parameter-Efficient Prompt Tuning (2021.emnlp-main)
Copied to clipboard
| Challenge: | Unlike discrete text prompts used by GPT-3, soft prompts are learned through backpropagation and can be tuned to incorporate signals from any number of labeled examples. |
| Approach: | They propose a mechanism for learning "soft prompts" to condition frozen language models to perform specific downstream tasks. |
| Outcome: | The proposed method outperforms fewshot learning using GPT-3 and matches the quality of model tuning as models exceed billions of parameters. |
Hard Prompts Made Interpretable: Sparse Entropy Regularization for Prompt Tuning with RL (2024.acl-long)
Copied to clipboard
Yunseon Choi, Sangmin Bae, Seonghyun Ban, Minchan Jeong, Chuheng Zhang, Lei Song, Li Zhao, Jiang Bian, Kee-Eung Kim
| Challenge: | Prompt tuning is an important technique for directing model behaviors and eliciting desired responses. |
| Approach: | They propose to find optimal prompt tokens using soft Q-learning to optimize models for prompt tuning. |
| Outcome: | The proposed method improves on baseline prompt tuning, and the results are more natural and interpretable. |
Prompt Tuning for Discriminative Pre-trained Language Models (2022.findings-acl)
Copied to clipboard
Yuan Yao, Bowen Dong, Ao Zhang, Zhengyan Zhang, Ruobing Xie, Zhiyuan Liu, Leyu Lin, Maosong Sun, Jianyong Wang
| Challenge: | Recent studies have shown promising results of prompt tuning in stimulating pre-trained language models (PLMs) for natural language processing tasks. |
| Approach: | They propose a prompt tuning framework that reformulates NLP tasks into a discriminative language modeling problem. |
| Outcome: | The proposed framework improves on text classification and question answering tasks and prevents unstable tuning problems in low-resource settings. |
StablePrompt : Automatic Prompt Tuning using Reinforcement Learning for Large Language Model (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in large language models have made it difficult to find appropriate prompts for tasks with multiple input-output formats. |
| Approach: | They propose a prompt tuning method based on reinforcement learning (RL) they propose an anchor model and an extension for generating input-dependent prompts. |
| Outcome: | The proposed method outperforms existing methods on a variety of tasks and achieves State-of-the-art performance across diverse types and sizes of LLMs. |
Contrastive Demonstration Tuning for Pre-trained Language Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies focus on searching discrete or continuous prompts or optimized verbalizers, yet the demonstration examples are crucial for an excellent final performance of prompt-tuning. |
| Approach: | They propose a pluggable, extensible, and efficient approach to prompt tuning that is free of demonstration sampling. |
| Outcome: | The proposed approach can be pluggable, extensible, and efficient on 16 datasets. |
Reinforcement Learning for Aligning Large Language Models Agents with Interactive Environments: Quantifying and Mitigating Prompt Overfitting (2025.findings-naacl)
Copied to clipboard
Mohamed Salim Aissi, Clément Romac, Thomas Carta, Sylvain Lamprier, Pierre-Yves Oudeyer, Olivier Sigaud, Laure Soulier, Nicolas Thome
| Challenge: | Reinforcement learning (RL) is a promising approach for aligning large language models knowledge with sequential decision-making tasks. |
| Approach: | They propose to use a contrastive loss framework to analyze the sensitivity of LLMs to prompt formulations following RL training in a textual environment. |
| Outcome: | The proposed framework improves the model's robustness and generalization capabilities by minimizing the model’s internal representations and salient tokens. |
Learning from Contrastive Prompts: An Automated Prompt Optimization Framework (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing prompt optimization methods often underperform due to learning exclusively from incorrect samples. |
| Approach: | They propose a framework that leverages contrastive prompts to distinguish between high- and low-performing cases. |
| Outcome: | The proposed framework can generalize across open and proprietary models and NLU benchmarks. |
SOPL: A Sequential Optimal Learning Approach to Automated Prompt Engineering in Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Using automated prompt engineering to identify effective features is essential for large language models. |
| Approach: | They propose an optimal learning framework for automated prompt engineering for black-box models . feature-based method is used to express prompt templates, which broadens the search space . |
| Outcome: | The proposed learning framework outperforms benchmark strategies on instruction induction tasks with limited budgets. |
RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning (2022.emnlp-main)
Copied to clipboard
Mingkai Deng, Jianyu Wang, Cheng-Ping Hsieh, Yihan Wang, Han Guo, Tianmin Shu, Meng Song, Eric Xing, Zhiting Hu
| Challenge: | Existing methods for finding the optimal prompt for a task are difficult to optimize. |
| Approach: | They propose an efficient discrete prompt optimization approach with reinforcement learning that generates the optimal discrete stimulus after training with reward. |
| Outcome: | The proposed approach is based on a parameter-efficient policy network that generates the optimal discrete prompt after training with reward. |
XPrompt: Exploring the Extreme of Prompt Tuning (2022.emnlp-main)
Copied to clipboard
| Challenge: | Prompt tuning learns soft prompts to condition pre-trained Language Models for performing downstream tasks in a parameter-efficient manner. |
| Approach: | They propose a Prompt tuning model with an eXtremely small scale that learns soft prompts to condition the frozen Pre-trained Language Models for performing downstream tasks in a parameter-efficient manner. |
| Outcome: | The proposed model outperforms the vanilla Prompt-Tuning and can significantly improve across tasks and model scales. |