EPO: Explicit Policy Optimization for Strategic Reasoning in LLMs via Reinforcement Learning (2025.acl-long)
Copied to clipboard
Xiaoqian Liu, Ke Wang, Yongbin Li, Yuchuan Wu, Wentao Ma, Aobo Kong, Fei Huang, Jianbin Jiao, Junge Zhang
| Challenge: | Existing methods for strategic reasoning face challenges in adaptability, scalability, and transferring strategies to new contexts. |
| Approach: | They propose an explicit policy optimization model that provides strategies in open-ended action space and can be plugged into arbitrary LLM agents to motivate goal-directed behavior. |
| Outcome: | The proposed model provides strategies in open-ended action space and can be plugged into arbitrary LLM agents to motivate goal-directed behavior. |
Similar Papers
Foresight Optimization for Strategic Reasoning in Large Language Models (2026.acl-long)
Copied to clipboard
Jessie Wang, Jiawen Duan, Jian Wang, Kaitao Song, Chunpu Xu, Johnny K. W. Ho, YU Fenggang, Johan F. Hoorn, Wenjie Li
| Challenge: | Existing reasoning enhancement methods do not capture foresight in LLMs. |
| Approach: | They propose to integrate opponent modeling principles into policy optimization to enhance strategic reasoning in LLMs by integrating opponent modeling into policy. |
| Outcome: | The proposed method outperforms existing reasoning-based LLMs in out-of-domain scenarios and shows that it significantly enhances strategic reasoning across LLM of varying sizes and origins. |
Agent-Pro: Learning to Evolve via Policy-Level Reflection and Optimization (2024.acl-long)
Copied to clipboard
Wenqi Zhang, Ke Tang, Hai Wu, Mengna Wang, Yongliang Shen, Guiyang Hou, Zeqi Tan, Peng Li, Yueting Zhuang, Weiming Lu
| Challenge: | Large Language Models (LLMs) are designed as specific task solvers with sophisticated prompt engineering, but are inherently incapacitating to address complex dynamic scenarios. |
| Approach: | They propose an LLM-based agent with policy-level reflection and optimization that can learn from interactive experiences and progressively elevate its behavioral policy. |
| Outcome: | The proposed agent outperforms vanilla LLM and specialized models in blackjack and Texas hold’em. |
Teaching LLMs to Plan, Not Just Solve: Plan Learning Boosts LLMs Generalization in Reasoning Tasks (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for reinforcement learning (RL) on self-generated data are limited in many domains. |
| Approach: | a new framework combines plan-based search with Step-level Advantage Preference Optimization to optimize plan learning. |
| Outcome: | The proposed framework improves in-domain performance and out-of-domain benchmarks. |
Enhancing Efficiency and Exploration in Reinforcement Learning for LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches allocate an equal number of rollouts to all questions during the RL process, which is inefficient. |
| Approach: | They propose a mechanism for dynamically allocating rollout budgets based on the difficulty of the problems, enabling more efficient RL training. |
| Outcome: | The proposed model improves response precision while preserving exploratory ability to uncover potential correct pathways. |
Teaching LLM to be Persuasive: Reward-Enhanced Policy Optimization for Alignment from Heterogeneous Rewards (2026.acl-industry)
Copied to clipboard
| Challenge: | a large language model (LLM) is used as a business development agent for persuasive price negotiation in online travel agencies. |
| Approach: | They propose a reward-enhancing policy optimization method that integrates three complementary reward sources-a preference-trained reward model and an LLM-as-a-judge. |
| Outcome: | The proposed method improves average dialogue rating to 4.63 (+0.33 over GRPO) and raises share of conversations with at least one excellent response to 66.67% (+23.34 pp over grepo). |
rSIM: Incentivizing Reasoning Capabilities of LLMs via Reinforced Strategy Injection (2026.acl-long)
Copied to clipboard
| Challenge: | Existing literature on Reasoning Language Models (RLMs) focuses on the ability to integrate reasoning strategies into the chain-of-thought process, contributing to improved problem-solving accuracy. |
| Approach: | They propose a reinforced strategy injection mechanism that enables any LLM to become an RLM by employing a small planner to guide the LLM's CoT through the adaptive injection of reasoning strategies. |
| Outcome: | The proposed model outperforms existing models in mathematical, coding, and financial reasoning tasks and is generalizable. |
Improving the Language Understanding Capabilities of Large Language Models Using Reinforcement Learning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Instruction-fine-tuned large language models (LLMs) under 14B parameters underperform on NLU tasks . we explore a framework to improve the NLU capabilities of LLMs . |
| Approach: | They propose to use Proximal Policy Optimization to improve NLU capabilities . they frame NLU as a reinforcement learning environment and optimize for reward signals . |
| Outcome: | The proposed framework outperforms supervised fine-tuning on GLUE and superGLUE tasks. |
Adapt to Thrive! Adaptive Power-Mean Policy Optimization for Improved LLM Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for reinforcement learning with verifiable rewards (RLVR) rely on static objective functions and rigid clipping strategies that misalign with the model’s evolving reasoning capabilities. |
| Approach: | They propose to incorporate Power-Mean Policy Optimization (PMPO) and Feedback-Adaptive Clipping (FAC) to overcome limitations of static mechanisms. |
| Outcome: | Extensive experiments on nine reasoning tasks show the proposed paradigm outperforms state-of-the-art methods. |
Can LLMs Reason in the Wild with Programs? (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models have shown superior capability to solve reasoning problems with programs. |
| Approach: | They propose a task where an LLM is tasked to solve a reasoning problem of unknown type by identifying the sub-problems and their corresponding formalisms. |
| Outcome: | The proposed model can be fine tuned to achieve better performance on ambiguous and mixed scope problems. |
Semantic-Aware Action Space Compression via LLM-DRL Synergy for Efficient Task-oriented Dialogue Policy Exploration (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained large language models (LLMs) with world knowledge and semantic understanding are promising for task-oriented dialogue systems. |
| Approach: | a framework that synergizes pre-trained large language models with DRL is proposed . a lightweight action pruning mechanism is employed to eliminate implausible actions . |
| Outcome: | a new framework synergizes pre-trained large language models with DRL to guide decision-making . the proposed framework eliminates semantically implausible or low-potential actions from multi-turn dialogue context . |