Warm Up Before You Train: Unlocking General Reasoning in Resource-Constrained Settings (2025.emnlp-main)
Copied to clipboard
| Challenge: | Reasoning-capable large language models (LLMs) have driven a major shift in artificial intelligence . these models generate long CoTs, capturing reasoning behaviors such as self-reflection, self-correction, and hypothesis testing. |
| Approach: | They propose a sample-efficient, two-stage training strategy to build reasoning LLMs . they "warm up" a model by distilling Long CoTs from a toy domain to acquire general reasoning skills . |
| Outcome: | The proposed training strategy outperforms existing models on a range of tasks. |
Similar Papers
From Data-Centric to Sample-Centric: Enhancing LLM Reasoning via Progressive Optimization (2026.acl-long)
Copied to clipboard
| Challenge: | Reinforcement learning with verifiable rewards (RLVR) has recently advanced the reasoning capabilities of large language models (LLMs). |
| Approach: | They propose a method that incorporates partial solution prefixes from expert demonstrations to guide the policy. |
| Outcome: | The proposed methods outperform strong baselines, yielding faster convergence and a higher performance ceiling. |
Knowledge-to-Verification: Exploring RLVR for LLMs in Knowledge-Intensive Domains (2026.acl-long)
Copied to clipboard
Zhonghang Yuan, Zhefan Wang, Fang Hu, Zihong Chen, Jinzhe Li, Gang Li, Jie Ying, Huanjun Kong, Songyang Zhang, Nanqing Dong
| Challenge: | Recent large language models (LLMs) have demonstrated remarkable progress in reasoning, but their applications on knowledge-intensive domains have not been explored due to the scarcity of high-quality verifiable data. |
| Approach: | They propose a framework that extends reinforcement learning with verifiable rewards (RLVR) to knowledge-intensive domains through automated verififiability data synthesis while enabling verification of the LLM's reasoning process. |
| Outcome: | Extensive experiments show that the proposed framework enhances the reasoning of large language models in knowledge-intensive domains without significantly compromising the model’s general capabilities. |
One Missing Piece for Open-Source Reasoning Models: A Dataset to Mitigate Cold-Starting Short CoT LLMs in RL (2025.acl-industry)
Copied to clipboard
Hyungjoo Chae, Dongjin Kang, Jihyuk Kim, Beong-woo Kwak, Sunghyun Park, Haeju Park, Jinyoung Yeo, Moontae Lee, Kyungjae Lee
| Challenge: | Existing large reasoning models are limited by their closed nature and high API costs and safety issues. |
| Approach: | They propose to build a long CoT dataset with existing short CoT LLMs that are not trained for inference-time scaling. |
| Outcome: | The proposed model achieves quality comparable to—or slightly below—R1 and is able to think longer and provide control over the thought budget to better manage the overthinking problem. |
EvoCoT: Overcoming the Exploration Bottleneck in Reinforcement Learning for LLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to reinforcement learning with verifiable reward (RLVR) are limited by difficulty or lack of exploration. |
| Approach: | They propose a self-evolving curriculum learning framework based on chain-of-thought reasoning optimization that constrains exploration space by self-generating and verifying CoT trajectories. |
| Outcome: | The proposed framework enables LLMs to solve previously unsolved problems without external supervision and is compatible with various RL fine-tuning methods. |
Beyond Reasoning Gains: Mitigating General-Capability Forgetting in Large Reasoning Models (2026.findings-acl)
Copied to clipboard
Hoang Phan, Xianjun Yang, Yuanshun Yao, Jingyu Zhang, Shengjie Bi, Xiaocheng Tang, Madian Khabsa, Lijuan Liu, Deren Lei
| Challenge: | Reinforcement learning with verifiable rewards (RLVR) has delivered impressive gains in mathematical and multimodal reasoning . however, the recipe introduces a significant risk of capability regression, where models forget foundational skills after prolonged training without employing regularization strategies. |
| Approach: | They propose a replay strategy with dynamic objective reweighting for general knowledge preservation using short-horizon signals of convergence and instability. |
| Outcome: | The proposed method preserves general capabilities and improves reasoning . it can be applied to existing RLVR pipelines without training additional models or tuning . |
S2R: Teaching LLMs to Self-verify and Self-correct via Reinforcement Learning (2025.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to incentivize LLMs’ deep thinking abilities require large-scale data or significant training efforts. |
| Approach: | They introduce an efficient framework that enhances LLM reasoning by teaching models to self-verify and self-correct during inference. |
| Outcome: | The proposed framework outperforms models trained on long-CoT distilled data with 3.1k initialization samples and achieves an accuracy improvement of 51.0% to 81.6%. |
Crossing the Reward Bridge: Expanding Reinforcement Learning with Verifiable Rewards Across Diverse Domains (2026.acl-long)
Copied to clipboard
| Challenge: | Reinforcement learning with verifiable rewards (RLVR) has been effective on structured tasks, but its reliance on simple, rule-based verifiers creates a bottleneck. |
| Approach: | They propose a framework that uses a generative verifier to provide soft, probabilistic rewards. |
| Outcome: | The proposed framework outperforms existing models up to 10x their size and can be scalable and effective. |
Bridging Reasoning and Action: Hybrid LLM–RL Framework for Efficient Cross-Domain Task-Oriented Dialogue (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to solve cross-domain task-oriented dialogues are brittle when cross- domain constraints are not directly grounded in surface text or require commonsense inference. |
| Approach: | They propose a framework that makes LLM-derived constraint reasoning usable for RL. |
| Outcome: | Experiments show that the proposed framework outperforms single-model baselines on long-horizon tasks. |
Correct, Concise and Complete: Multi-stage Training For Adaptive Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) increase test-time computation, often in the form of chain-of-thought (CoT) however, reasoning traces can become unnecessarily long, increasing computation costs without improving accuracy and sometimes even degrading performance. |
| Approach: | They propose a multi-stage efficient reasoning method that combines supervised fine-tuning with reinforcement learning using an adaptive length penalty. |
| Outcome: | The proposed method reduces response length by an average of 28% for 8B models and 40% for 32B models while incurring only minor performance drops of 1.6 and 2.5 points, respectively. |
Better LLM Reasoning via Dual-Play (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have made remarkable progress through Reinforcement Learning with Verifiable Rewards (RLVR) however, external supervision remains a bottleneck for tasks and domains for which supervised data are scarce or non-existent. |
| Approach: | They propose a novel dual-play framework that adversarially trains two models initialized from the same base model. |
| Outcome: | The proposed framework improves the math reasoning performance of large language models. |