Semantic-Space Exploration and Exploitation in RLVR for LLM Reasoning (2026.findings-acl)
Copied to clipboard
Fanding Huang, Guanbo Huang, Xiao Fan, Yi He, Xiao Liang, Xiao Chen, Qinting Jiang, Faisal Nadeem Khan, Jingyan Jiang, Zhi Wang
| Challenge: | Recent advances in Reinforcement Learning with Verifiable Rewards (RLVR) have substantially improved the reasoning abilities of Large Language Models (LLMs). |
| Approach: | They propose a method that balances exploration and exploitation in the hidden-state space of response trajectories. |
| Outcome: | The proposed model yields consistent improvements across models, algorithms and reasoning benchmarks. |
Similar Papers
Unlocking Exploration in RLVR: Uncertainty-aware Advantage Shaping for Deeper Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Reinforcement Learning with Verifiable Rewards (RLVR) has shown significant promise for enhancing the reasoning capabilities of large language models (LLMs). |
| Approach: | They propose a model-free method that refines credit assignment by leveraging the model's internal uncertainty signals. |
| Outcome: | Extensive experiments on five mathematical reasoning benchmarks show that the proposed method outperforms strong RLVR baselines on multiple model scales, including 1.5B and 7B. |
Knowledge-to-Verification: Exploring RLVR for LLMs in Knowledge-Intensive Domains (2026.acl-long)
Copied to clipboard
Zhonghang Yuan, Zhefan Wang, Fang Hu, Zihong Chen, Jinzhe Li, Gang Li, Jie Ying, Huanjun Kong, Songyang Zhang, Nanqing Dong
| Challenge: | Recent large language models (LLMs) have demonstrated remarkable progress in reasoning, but their applications on knowledge-intensive domains have not been explored due to the scarcity of high-quality verifiable data. |
| Approach: | They propose a framework that extends reinforcement learning with verifiable rewards (RLVR) to knowledge-intensive domains through automated verififiability data synthesis while enabling verification of the LLM's reasoning process. |
| Outcome: | Extensive experiments show that the proposed framework enhances the reasoning of large language models in knowledge-intensive domains without significantly compromising the model’s general capabilities. |
Reinforced Efficient Reasoning via Semantically Diverse Exploration (2026.acl-long)
Copied to clipboard
Ziqi Zhao, Zhaochun Ren, Jiahong Zou, Liu Yang, Zhiwei Xu, Xuri Ge, Zhumin Chen, Xinyu Ma, Daiting Shi, Shuaiqiang Wang, Dawei Yin, Xin Xin
| Challenge: | Existing methods for reinforcement learning with verifiable rewards suffer from limited exploration diversity and inefficient reasoning. |
| Approach: | They propose a method that rewards concise and correct reasoning while penalizing unnecessarily long reasoning chains. |
| Outcome: | Extensive experiments on Qwen and Llama models validate the effectiveness and efficiency of ROSE. |
Crossing the Reward Bridge: Expanding Reinforcement Learning with Verifiable Rewards Across Diverse Domains (2026.acl-long)
Copied to clipboard
| Challenge: | Reinforcement learning with verifiable rewards (RLVR) has been effective on structured tasks, but its reliance on simple, rule-based verifiers creates a bottleneck. |
| Approach: | They propose a framework that uses a generative verifier to provide soft, probabilistic rewards. |
| Outcome: | The proposed framework outperforms existing models up to 10x their size and can be scalable and effective. |
VANE: Guiding High-Value Exploration in RLVR via Outcome-Process Novelty Shaping (2026.findings-acl)
Copied to clipboard
| Challenge: | Extensive experiments on large-scale mathematical reasoning and out-of-distribution tasks demonstrate the effectiveness and generalization of the proposed method. |
| Approach: | They propose a method that quantifies novelty across the outcome space and semantic process space by using reward or solution divergence. |
| Outcome: | Experiments on Qwen2.5-Math-7B demonstrate the proposed method is general and efficient. |
Rethinking Entropy Interventions in RLVR: An Entropy Change Perspective (2026.acl-long)
Copied to clipboard
Zhezheng Hao, Hong Wang, Haoyang Liu, Jian Luo, Jiarui Yu, Hande Dong, Qiang Lin, Can Wang, Jiawei Chen
| Challenge: | Large Language Models (LLMs) have remarkable reasoning capabilities in complex tasks such as mathematics and coding. |
| Approach: | They propose an entropy-modulation method that adaptively reweighs tokens based on theoretically-estimated entropic variations. |
| Outcome: | The proposed method outperforms state-of-the-art methods in six mathematical reasoning and three coding benchmarks. |
RL-PLUS: Countering Capability Boundary Collapse of LLMs in Reinforcement Learning with Hybrid-policy Optimization (2026.acl-long)
Copied to clipboard
Yihong Dong, Xue Jiang, Yongding Tao, Huanyu Liu, Kechi Zhang, Lili Mou, Rongyu Cao, Yingwei MA, Jue Chen, Binhua Li, Zhi Jin, Fei Huang, Yongbin Li, Ge Li
| Challenge: | Reinforcement Learning with Verifiable Reward (RLVR) has significantly advanced the complex reasoning abilities of Large Language Models (LLMs). |
| Approach: | They propose a hybrid-policy optimization approach that synergizes internal exploitation with external data to achieve stronger reasoning capabilities. |
| Outcome: | The proposed approach achieves state-of-the-art performance on six math reasoning benchmarks and superior performance on out-of distribution reasoning tasks. |
Beyond High-Entropy Exploration: Correctness-Aware Low-Entropy Segment-Based Advantage Shaping for Reasoning LLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent work studies RLVR through token entropy, arguing that high-entropies drive exploration and should receive stronger updates. |
| Approach: | They propose a correctness-aware reinforcement framework that performs fine-grained advantage modulation over low-entropy segments. |
| Outcome: | The proposed framework improves accuracy over strong RL baselines across three backbones and six math benchmarks while maintaining high-entropy exploration. |
Revisiting Entropy in Reinforcement Learning for Large Reasoning Models (2026.findings-acl)
Copied to clipboard
Renren Jin, Pengzhi Gao, Yuqi Ren, Zhuowen Han, Tongxuan Zhang, Wuwei Huang, Wei Liu, Jian Luan, Deyi Xiong
| Challenge: | Reinforcement learning with verifiable rewards (RLVR) has emerged as a paradigm for enhancing the reasoning capabilities of large language models. |
| Approach: | They propose a positive-advantage reweighting approach that regulates model entropy by adjusting the loss weights assigned to tokens with positive advantages during RLVR training. |
| Outcome: | The proposed approach regulates model entropy by adjusting loss weights assigned to tokens with positive advantages during RLVR training while maintaining competitive performance. |
Revisiting Entropy Regularization: Adaptive Coefficient Unlocks Its Potential for LLM Reinforcement Learning (2026.findings-acl)
Copied to clipboard
| Challenge: | Reasoning ability is a defining capability of Large Language Models (LLMs), but RLVR training suffers from policy entropy collapse, hindering exploration and limiting reasoning performance. |
| Approach: | They propose a framework that dynamically balances exploration and exploitation via three components: difficulty-aware coefficient allocation, initial-anchored target entropy, and dynamic global coefficient adjustment. |
| Outcome: | The proposed framework outperforms baselines on multiple mathematical reasoning benchmarks. |