Thinking-Based Non-Thinking: Solving the Reward Hacking Problem in Training Hybrid Reasoning Models via Reinforcement Learning (2026.acl-long)
Copied to clipboard
Siyuan Gan, Jiaheng Liu, Boyan Wang, Tianpei Yang, Runqing Miao, Yuyao Zhang, Fanyu Meng, Junlan Feng, Linjian Meng, Jing Huo, Yang Gao
| Challenge: | Existing work on large reasoning models (LRMs) focuses on using reinforcement learning (RL) to train hybrid reasoning models that automatically decide whether to engage in thinking or not based on the complexity of the query. |
| Approach: | They propose to use reinforcement learning to train hybrid reasoning models that automatically decide whether to engage in thinking or not based on the complexity of the query. |
| Outcome: | The proposed model reduces token usage by around 50%$ compared to DeepSeek-R1-Distill-Qwen-1.5B/7B and DeepScaleR-1.5b, while significantly improving accuracy. |
Similar Papers
AdaptThink: Reasoning Models Can Learn When to Think (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in large reasoning models have demonstrated remarkable capabilities in tackling complex tasks. |
| Approach: | They propose an algorithm to teach reasoning models to choose the optimal thinking mode based on problem difficulty. |
| Outcome: | The proposed algorithm reduces the average response length and improves accuracy on three math datasets. |
Skip-Thinking: Chunk-wise Chain-of-Thought Distillation Enable Smaller Language Models to Reason Better and Faster (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods train small language models to learn long rationales in one iteration. |
| Approach: | They propose a method that uses a heuristic search to divide rationale into internal chunks . they propose CWT, which uses CWt to focus SLM on learning from only one chunk per iteration. |
| Outcome: | The proposed method can guide a large language model (LLM) in reasoning tasks. |
Think How to Think: Mitigating Overthinking with Autonomous Difficulty Cognition in Large Reasoning Models (2026.acl-long)
Copied to clipboard
| Challenge: | Recent Large Reasoning Models (LRMs) excel at complex reasoning tasks but often suffer from overthinking. |
| Approach: | They propose a two-stage fine-tuning strategy that progressively inspires LRMs’ difficulty cognition and redundancy cognition of LRM. |
| Outcome: | The proposed model significantly reduces inference costs by over 70% on easy tasks and 40% on complex ones without compromising performance. |
When to Continue Thinking: Adaptive Thinking Mode Switching for Efficient Reasoning (2025.findings-emnlp)
Copied to clipboard
Xiaoyun Zhang, Jingqing Ruan, Xing Ma, Yawen Zhu, Haodong Zhao, Hao Li, Jiansong Chen, Ke Zeng, Xunliang Cai
| Challenge: | Large reasoning models (LRMs) incur excessive computational overhead due to redundant reasoning, especially on simple tasks. |
| Approach: | They propose an Adaptive Self-Recovery Reasoning framework that suppresses unnecessary reasoning and enables implicit recovery. |
| Outcome: | The proposed framework suppresses unnecessary reasoning and enables implicit recovery. |
Mid-Think: Training-Free Intermediate-Budget Reasoning via Token-Level Triggers (2026.findings-acl)
Copied to clipboard
Van Yang, Shouren Wang, Debargha Ganguly, Xinpeng Li, Chaoda Song, Vikash Singh, Vipin Chaudhary, Xiaotian Han
| Challenge: | Explicit /think> tags are used to expose intermediate reasoning and enable hybrid thinking behaviors. |
| Approach: | They propose a training-free prompting format that combines these triggers to achieve intermediate-budget reasoning, outperforming fixed-token and prompt-based baselines in terms of the accuracy–length trade-off. |
| Outcome: | The proposed method outperforms fixed-token and prompt-based prompts in accuracy–length trade-offs while improving Qwen3-8B on AIME from 69.8% to 72.4% and on GPQA from 58.5% to 61.1%. |
Correct, Concise and Complete: Multi-stage Training For Adaptive Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) increase test-time computation, often in the form of chain-of-thought (CoT) however, reasoning traces can become unnecessarily long, increasing computation costs without improving accuracy and sometimes even degrading performance. |
| Approach: | They propose a multi-stage efficient reasoning method that combines supervised fine-tuning with reinforcement learning using an adaptive length penalty. |
| Outcome: | The proposed method reduces response length by an average of 28% for 8B models and 40% for 32B models while incurring only minor performance drops of 1.6 and 2.5 points, respectively. |
Fast Quiet-STaR: Thinking Without Thought Tokens (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models have achieved impressive performance across a range of tasks, but further gains require more than scaling up model sizes or training data. |
| Approach: | They propose a method that gradually reduces the number of thought tokens . this method allows models to internalize more abstract reasoning processes . |
| Outcome: | The proposed framework preserves the benefits of token-level reasoning while reducing computational cost. |
ThinkSwitcher: When to Think Hard, When to Think Fast (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large reasoning models excel at solving complex tasks by leveraging long chain-of-thought (CoT) reasoning. |
| Approach: | They propose a framework that enables a single LRM to dynamically switch between short and long CoT modes based on task complexity. |
| Outcome: | The proposed framework reduces computational cost by 20-30% while maintaining high accuracy on complex tasks. |
How Do Humans Write Code? Large Models Do It the Same Way Too (2024.emnlp-main)
Copied to clipboard
| Challenge: | Program-of-Thought (PoT) replaces natural language-based Chain-ofThough (CoT) but introduces more reasoning errors, such as incorrect formulas or flawed logic, compared to CoT. |
| Approach: | They propose a method that integrates CoT and Program-of-Thought to achieve more accurate reasoning and reinforcement learning. |
| Outcome: | The proposed method achieves an average improvement of 6.5% on the Llama-Base model and 4.3% on the Mistral-Bass model across 8 mathematical calculation datasets. |
RiT: Rubrics-in-Thinking Reinforcement Learning for Improved Reasoning in Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Reasoning Models benefit from generating intermediate reasoning steps alongside final answers. |
| Approach: | They propose a framework to introduce thinking-rubric supervision into intermediate reasoning. |
| Outcome: | The proposed framework outperforms outcome-only RL baselines on reasoning-intensive and open-ended tasks. |