CARL: Constraint-Aware Reinforcement Learning for Planning with LLMs (2026.findings-acl)
Copied to clipboard
Qiuyi Qi, Jinjian Zhang, Mutian Bao, Tian Liang, Guocong Li, Dongnan Liu, Wei Zhou, Jie Liu, Ming Kong, Linjian Mo, Feng Zhang, Qiang Zhu
| Challenge: | Existing approaches to constraint-aware planning fail to enhance the model’s intrinsic focus on constraints. |
| Approach: | They propose a constraint-aware reinforcement learning framework that encourages constraint focus and penalizes neglect of LLMs. |
| Outcome: | The proposed framework outperforms existing frameworks and state-of-the-art reasoning models in a number of real-world applications. |
Similar Papers
Teaching LLMs to Plan, Not Just Solve: Plan Learning Boosts LLMs Generalization in Reasoning Tasks (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for reinforcement learning (RL) on self-generated data are limited in many domains. |
| Approach: | a new framework combines plan-based search with Step-level Advantage Preference Optimization to optimize plan learning. |
| Outcome: | The proposed framework improves in-domain performance and out-of-domain benchmarks. |
Encouraging Good Processes Without the Need for Good Answers: Reinforcement Learning for LLM Agent Planning (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Currently, the dominant end-to-end reinforcement learning paradigm for agents in Large Language Models (LLMs) employs multi-objective optimization that jointly trains both planning and answer summarization capabilities. |
| Approach: | They propose a framework that decouples the training process to enable a focused, single-objective optimization of the planning module. |
| Outcome: | The proposed framework achieves an 8%–12% improvement in planning performance compared to end-to-end baselines. |
Enhancing Efficiency and Exploration in Reinforcement Learning for LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches allocate an equal number of rollouts to all questions during the RL process, which is inefficient. |
| Approach: | They propose a mechanism for dynamically allocating rollout budgets based on the difficulty of the problems, enabling more efficient RL training. |
| Outcome: | The proposed model improves response precision while preserving exploratory ability to uncover potential correct pathways. |
A Survey of Reinforcement Learning for Large Language Models under Data Scarcity: Challenges and Solutions (2026.acl-long)
Copied to clipboard
Zhiyin Yu, Yuchen Mou, Juncheng Yan, Junyu Luo, Chunchun Chen, Xing Wei, Yunhui Liu, Hongru Sun, Yuxing Zhang, Jun Xu, Yatao Bian, Ming Zhang, Wei Ye, Tieke He, Jie Yang, Guanjie Zheng, Zhonghai Wu, Bo Zhang, Lei Bai, Xiao Luo
| Challenge: | Existing research on reinforcement learning for LLMs under data scarcity has not been unified. |
| Approach: | They propose a top-up hierarchical framework built around three complementary perspectives: data-centric, training-centric and framework-centric. |
| Outcome: | The proposed framework provides a clear conceptual foundation for understanding the design space of data-efficient RL for large language models and to guide researchers working in this emerging area. |
Inverse Reinforcement Learning Meets Large Language Model Alignment (2025.acl-tutorials)
Copied to clipboard
| Challenge: | This tutorial will provide a comprehensive review of recent advances in LLM alignment . it will highlight the necessity of constructing neural reward models from human data . |
| Approach: | This tutorial will provide a comprehensive review of recent advances in LLM alignment through the lens of inverse reinforcement learning. |
| Outcome: | This tutorial will provide a comprehensive review of recent advances in LLM alignment through the lens of inverse reinforcement learning (IRL). |
CaRL-EM: Cost-Aware Reinforcement Learning for Entity Matching with LLMs (2026.acl-long)
Copied to clipboard
| Challenge: | Entity matching (EM) requires fine-grained contextual understanding and domain knowledge. |
| Approach: | They propose a reinforcement learning controller that manages LLM operations by combining multiple operators and a set of model capacities. |
| Outcome: | The proposed controller can be reused with different LLM backends at inference time without retraining. |
Forge: Quality-Aware Reinforcement Learning for NP-Hard Optimization in LLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks focus on correctness, overlooking optimality . large language models excel at math, coding, logic and puzzles . |
| Approach: | They propose a framework for training and evaluating Large Language Models on NP-hard optimization problems through quality-aware RLVR. |
| Outcome: | The proposed framework outperforms existing benchmarks on math, coding, logic and puzzles. |
Sparse-RL: Breaking the Memory Wall in LLM Reinforcement Learning via Stable Sparse Rollouts (2026.acl-long)
Copied to clipboard
Sijia Luo, Xiaokang Zhang, Yuxuan Hu, Bohan Zhang, Ke Wang, Jinbo Su, Mengshu Sun, Lei Liang, Jing Zhang
| Challenge: | Existing methods for storing key-value caches during long-horizon rollouts cause performance collapses. |
| Approach: | They propose a new training paradigm that empowers stable RL training under sparse rollouts. |
| Outcome: | The proposed model reduces rollout overhead while maintaining the performance. |
RL-PLUS: Countering Capability Boundary Collapse of LLMs in Reinforcement Learning with Hybrid-policy Optimization (2026.acl-long)
Copied to clipboard
Yihong Dong, Xue Jiang, Yongding Tao, Huanyu Liu, Kechi Zhang, Lili Mou, Rongyu Cao, Yingwei MA, Jue Chen, Binhua Li, Zhi Jin, Fei Huang, Yongbin Li, Ge Li
| Challenge: | Reinforcement Learning with Verifiable Reward (RLVR) has significantly advanced the complex reasoning abilities of Large Language Models (LLMs). |
| Approach: | They propose a hybrid-policy optimization approach that synergizes internal exploitation with external data to achieve stronger reasoning capabilities. |
| Outcome: | The proposed approach achieves state-of-the-art performance on six math reasoning benchmarks and superior performance on out-of distribution reasoning tasks. |
Efficient Integration of External Knowledge to LLM-based World Models via Retrieval-Augmented Generation and Reinforcement Learning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing attempts to enhance LLM-based world models through prompting or fine-tuning approaches are either requiring human knowledge or computationally extensive. |
| Approach: | They propose a framework that leverages retrieval-augmented generation to integrate external knowledge to LLM-based world models. |
| Outcome: | The proposed framework outperforms baseline models and exhibits strong generalizability. |