Prompt-Based Length Controlled Generation with Multiple Control Types (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing length control methods focus on a simple control type of “equal to” a target length. |
| Approach: | They propose a prompt-based method to achieve length controlled generation under different control types with high accuracy by using reinforcement learning and sample filtering with the reward signal given by rule-based reward models. |
| Outcome: | The proposed method significantly improves the accuracy of prompt-based length control on popular summarization datasets like CNNDM and NYT under multiple control types. |
Similar Papers
DcLM: Output Length Control of Large Language Models via Dynamic Length Markers (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have limited awareness of output length, making it difficult to satisfy precise length requirements. |
| Approach: | They propose a model-agnostic approach that introduces dynamic length markers to guide length-controllable outputs. |
| Outcome: | The proposed method significantly reduces length deviation across multiple datasets. |
Length Controlled Generation for Black-box LLMs (2025.acl-long)
Copied to clipboard
Yuxuan Gu, Wenjie Wang, Xiaocheng Feng, Weihong Zhong, Kun Zhu, Lei Huang, Ting Liu, Bing Qin, Tat-Seng Chua
| Challenge: | Existing length control methods involve fine-tuning the parameters of LLMs, which is inefficient and suboptimal for practical use. |
| Approach: | They propose an iterative sampling framework that regulates LLMs to generate length-constrained text without modifying the underlying parameters. |
| Outcome: | The proposed method achieves 100% success rates on Llama3.1 tasks with minimal additional computational overhead. |
Removing Prompt-template Bias in Reinforcement Learning from Human Feedback (2025.findings-acl)
Copied to clipboard
| Challenge: | Reinforcement Learning from Human Feedback (RLHF) has shown promise for enhancing pre-trained large language models to generate responses that align with human preferences and societal values. |
| Approach: | They propose a method to estimate prompt-template bias term during reward modeling and use it to calibrate reward scores. |
| Outcome: | The proposed method can be flexibly combined with existing algorithms of removing length bias, leading to a further improvement in the aspect of enhancing the quality of generated responses. |
Controlled Generation with Prompt Insertion for Natural Language Explanations in Grammatical Error Correction (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing studies present tokens, examples, and hints for corrections, but do not directly explain the reasons in natural language. |
| Approach: | They propose a method called controlled generation with Prompt Insertion that uses Large Language Models to explain the reasons for corrections in natural language. |
| Outcome: | The proposed method can explain the reasons for corrections in natural language by guiding the LLMs to generate explanations for all correction points. |
Control Large Language Models via Divide and Conquer (2024.emnlp-main)
Copied to clipboard
| Challenge: | Lexically Constrained Generation (LCG) is a crucial task of text generation. |
| Approach: | They propose a Divide and Conquer Generation strategy to enhance LLMs' performance in Lexically Constrained Generation with prompt-based controlling. |
| Outcome: | The proposed strategy shows 90% improvement on the most challenging LCG task. |
Prompt-R1: Collaborative Automatic Prompting Framework via End-to-end Reinforcement Learning (2026.findings-acl)
Copied to clipboard
Wenjin Liu, Haoran Luo, Xueyuan Lin, Haoming Liu, Tiesunlong Shen, Jiapu Wang, Rui Mao, Erik Cambria
| Challenge: | Existing large language models are limited in understanding, reasoning, calculation, and generation, limiting their performance in complex reasoning and dynamic tasks. |
| Approach: | They propose a plug-and-play framework that integrates a small-scale LLM (as agent) with large-scale large-level LLMs (a as environment) they propose generating prompts that are used to interact with LLM, and a double constraint reward that optimizes correctness and quality of generation. |
| Outcome: | The proposed framework significantly outperforms baseline large-scale large-language models across various tasks. |
Mixture of Soft Prompts for Controllable Data Generation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) generate fluent text when the target output follows natural language patterns. |
| Approach: | They propose a method that uses large language models to generate fluent text from a limited ontology rather than direct prediction by using soft prompts. |
| Outcome: | The proposed method produces diverse and natural text while preserving label semantics. |
RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning (2022.emnlp-main)
Copied to clipboard
Mingkai Deng, Jianyu Wang, Cheng-Ping Hsieh, Yihan Wang, Han Guo, Tianmin Shu, Meng Song, Eric Xing, Zhiting Hu
| Challenge: | Existing methods for finding the optimal prompt for a task are difficult to optimize. |
| Approach: | They propose an efficient discrete prompt optimization approach with reinforcement learning that generates the optimal discrete stimulus after training with reward. |
| Outcome: | The proposed approach is based on a parameter-efficient policy network that generates the optimal discrete prompt after training with reward. |
CRL-Prompt: Contrastive and Reinforcement Learning for Soft Prompt Tuning for Text Classification (2026.acl-srw)
Copied to clipboard
| Challenge: | Manual prompt engineering is time-consuming, non-scalable, and brittle, while current auto-prompting techniques are far from maturity. |
| Approach: | They propose a two-stage method for prompt learning of frozen language models, CRL-Prompt, based on soft prompt initialization followed by contrastive and reinforcement-based refinement. |
| Outcome: | The proposed method achieves consistent improvements over baseline prompt tuning strategies, with gains of up to 2.2% while training fewer than 0.25% of model parameters. |
Fixed Input Parameterization for Efficient Prompting (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have shown that attaching prompts to the input is effective at conditioning Language Models (LMs) however, prompts are always included in the input text during inference, thus incurring substantial computational and memory overhead. |
| Approach: | They propose to inject a fixed prompt into the parameters of an LM to be an efficient alternative to attaching fixed prompts to the input. |
| Outcome: | The proposed method can be up to 280 times more efficient in terms of FLOPs than previous approaches. |