Reinforced Efficient Reasoning via Semantically Diverse Exploration (2026.acl-long)
Copied to clipboard
Ziqi Zhao, Zhaochun Ren, Jiahong Zou, Liu Yang, Zhiwei Xu, Xuri Ge, Zhumin Chen, Xinyu Ma, Daiting Shi, Shuaiqiang Wang, Dawei Yin, Xin Xin
| Challenge: | Existing methods for reinforcement learning with verifiable rewards suffer from limited exploration diversity and inefficient reasoning. |
| Approach: | They propose a method that rewards concise and correct reasoning while penalizing unnecessarily long reasoning chains. |
| Outcome: | Extensive experiments on Qwen and Llama models validate the effectiveness and efficiency of ROSE. |
Similar Papers
SD-E2: Semantic Exploration for Reasoning Under Token Budgets (2026.findings-eacl)
Copied to clipboard
| Challenge: | Small language models struggle with complex reasoning because exploration is expensive under tight compute budgets. |
| Approach: | They propose a framework that makes exploration explicit by optimizing semantic diversity in generated reasoning trajectories. |
| Outcome: | The proposed framework surpasses Qwen2.5-3B-Instruct and strong GRPO baselines on GSM8K and improves on the harder AIME benchmark to 13.28% vs. base 6.74%. |
Beyond High-Entropy Exploration: Correctness-Aware Low-Entropy Segment-Based Advantage Shaping for Reasoning LLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent work studies RLVR through token entropy, arguing that high-entropies drive exploration and should receive stronger updates. |
| Approach: | They propose a correctness-aware reinforcement framework that performs fine-grained advantage modulation over low-entropy segments. |
| Outcome: | The proposed framework improves accuracy over strong RL baselines across three backbones and six math benchmarks while maintaining high-entropy exploration. |
Unlocking Exploration in RLVR: Uncertainty-aware Advantage Shaping for Deeper Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Reinforcement Learning with Verifiable Rewards (RLVR) has shown significant promise for enhancing the reasoning capabilities of large language models (LLMs). |
| Approach: | They propose a model-free method that refines credit assignment by leveraging the model's internal uncertainty signals. |
| Outcome: | Extensive experiments on five mathematical reasoning benchmarks show that the proposed method outperforms strong RLVR baselines on multiple model scales, including 1.5B and 7B. |
Enhancing Efficiency and Exploration in Reinforcement Learning for LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches allocate an equal number of rollouts to all questions during the RL process, which is inefficient. |
| Approach: | They propose a mechanism for dynamically allocating rollout budgets based on the difficulty of the problems, enabling more efficient RL training. |
| Outcome: | The proposed model improves response precision while preserving exploratory ability to uncover potential correct pathways. |
EvoCoT: Overcoming the Exploration Bottleneck in Reinforcement Learning for LLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to reinforcement learning with verifiable reward (RLVR) are limited by difficulty or lack of exploration. |
| Approach: | They propose a self-evolving curriculum learning framework based on chain-of-thought reasoning optimization that constrains exploration space by self-generating and verifying CoT trajectories. |
| Outcome: | The proposed framework enables LLMs to solve previously unsolved problems without external supervision and is compatible with various RL fine-tuning methods. |
Semantic-Space Exploration and Exploitation in RLVR for LLM Reasoning (2026.findings-acl)
Copied to clipboard
Fanding Huang, Guanbo Huang, Xiao Fan, Yi He, Xiao Liang, Xiao Chen, Qinting Jiang, Faisal Nadeem Khan, Jingyan Jiang, Zhi Wang
| Challenge: | Recent advances in Reinforcement Learning with Verifiable Rewards (RLVR) have substantially improved the reasoning abilities of Large Language Models (LLMs). |
| Approach: | They propose a method that balances exploration and exploitation in the hidden-state space of response trajectories. |
| Outcome: | The proposed model yields consistent improvements across models, algorithms and reasoning benchmarks. |
Do Not Step Into the Same River Twice: Learning to Reason from Trial and Error (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to RLVR train LMs based on their own on-policy responses and are constrained by the initial capability of LM. |
| Approach: | They propose an approach that hints LMs with their self-made mistakes without external guidance. |
| Outcome: | The proposed approach outperforms the normal group relative policy optimization and requires no external guidance. |
AIPO: Adaptive Information Guided Token-Level Reinforcement Learning for Large Language Model Reasoning (2026.acl-long)
Copied to clipboard
| Challenge: | Existing RLVR methods focus on all generated tokens rather than on which tokens contribute to reasoning. |
| Approach: | They propose to use a Random–Fourier approximation of the Hilbert–Schmidt Independence Criterion to focus updates on decisive tokens discovered on the fly to improve the efficiency of mutual-information estimation. |
| Outcome: | The proposed approach yields +20% accuracy over strong RLVR baselines while updating merely 10% of tokens, demonstrating superior efficiency and effectiveness. |
LLaMA-Berry: Pairwise Optimization for Olympiad-level Mathematical Reasoning via O1-like Monte Carlo Tree Search (2025.naacl-long)
Copied to clipboard
Di Zhang, Jianbo Wu, Jingdi Lei, Tong Che, Jiatong Li, Tong Xie, Xiaoshui Huang, Shufei Zhang, Marco Pavone, Yuqiang Li, Wanli Ouyang, Dongzhan Zhou
| Challenge: | LLaMA-Berry is an advanced mathematical reasoning framework to enhance the problem-solving ability of large language models (LLMs). |
| Approach: | They propose a Monte Carlo Tree Search and Self-Refine framework to optimize reasoning paths and a pairwise reward model to evaluate different paths globally. |
| Outcome: | The proposed framework overcomes inefficiencies and limitations of step-wise and greedy search algorithms, enabling more efficient exploration of solution spaces. |
Crossing the Reward Bridge: Expanding Reinforcement Learning with Verifiable Rewards Across Diverse Domains (2026.acl-long)
Copied to clipboard
| Challenge: | Reinforcement learning with verifiable rewards (RLVR) has been effective on structured tasks, but its reliance on simple, rule-based verifiers creates a bottleneck. |
| Approach: | They propose a framework that uses a generative verifier to provide soft, probabilistic rewards. |
| Outcome: | The proposed framework outperforms existing models up to 10x their size and can be scalable and effective. |