| Challenge: | Reasoning LLMs often spend tokens on long intermediate reasoning traces when solving new problems. |
| Approach: | They propose to store reusable reasoning skills distilled from extensive deliberation and trial-and-error exploration and retrieve these skills at inference time to guide future reasoning. |
| Outcome: | The proposed approach reduces reasoning tokens while improving overall performance on coding and mathematical reasoning tasks. |
Similar Papers
Not All Tokens Matter: Towards Efficient LLM Reasoning via Token Significance in Reinforcement Learning (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) often produce unnecessarily long explanations that reduce efficiency. |
| Approach: | They propose a length-aware reward that selectively penalizes insignificance tokens . they also propose 'dynamic length control' that encourages more detailed reasoning . |
| Outcome: | The proposed method reduces response length while maintaining correctness, the authors show . it selectively penalizes insignificance tokens while maintaining accuracy . |
Token-Budget-Aware LLM Reasoning (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to enhance reasoning capabilities of large language models incur significant overhead in token usage, leading to increased costs. |
| Approach: | They propose a token-budget-aware LLM reasoning framework that adjusts the number of reasoning tokens based on the reasoning complexity of each problem. |
| Outcome: | The proposed method reduces token costs in CoT reasoning with only a slight performance reduction. |
Do LLMs Really Need 10+ Thoughts for “Find the Time 1000 Days Later”? Towards Structural Understanding of LLM Overthinking (2026.acl-long)
Copied to clipboard
Xinliang Frederick Zhang, Anhad Mohananey, Alexandra Chronopoulou, Pinelopi Papalampidi, Somit Gupta, Tsendsuren Munkhdalai, Lu Wang, Shyam Upadhyay
| Challenge: | Existing studies on LLMs' thought processes are limited to superficial, profiling-based observations, failing to delve into their inner workings. |
| Approach: | They propose a utility-based definition of overthinking that moves beyond length-based metrics and provides a more insightful understanding of LLMs' thought progression. |
| Outcome: | The proposed model decomposes the LLM thought process into minimally complete sub-thoughts and identifies common thinking patterns for topically similar queries. |
Current Advances in LLM Reasoning (2026.acl-tutorials)
Copied to clipboard
| Challenge: | This tutorial examines comprehensive evaluation strategies to assess the reasoning abilities of large language models (LLMs) advanced inference time methods and post-training methods that aim to make LLMs think more like humans are discussed in this tutorial. |
| Approach: | This tutorial explores comprehensive evaluation strategies to assess the reasoning abilities of large language models (LLMs) and discusses two types of methods to improve models’ reasoning: advanced inference time methods, structured and self-improvement inference methods, and post-training methods, such as RLHF, DPO, and GRPO. |
| Outcome: | This tutorial examines evaluation strategies to assess the reasoning abilities of large language models and discusses two types of methods to improve models’ reasoning. |
Beyond Meta-Reasoning: Metacognitive Consolidation for Self-Improving LLM Reasoning (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches for improving LLM reasoning remain episodic and lack reusable meta-reasoning skills. |
| Approach: | They propose a framework that consolidates metacognitive experience from past reasoning episodes into reusable knowledge that improves future meta-reasoning. |
| Outcome: | The proposed framework consolidates metacognitive experience from past reasoning episodes into reusable knowledge that improves future meta-reasoning. |
Think Clearly: Improving Reasoning via Redundant Token Pruning (2025.findings-emnlp)
Copied to clipboard
Daewon Choi, Jimin Lee, Jihoon Tack, Woomin Song, Saket Dingliwal, Sai Muralidhar Jayanthi, Bhavana Ganesh, Jinwoo Shin, Aram Galstyan, Sravan Babu Bodapati
| Challenge: | Recent large language models show promising capabilities in long-form reasoning . however, they tend to include substantial redundancy in reasoning paths . |
| Approach: | They propose a structure-aware pruning method that prioritizes removing redundant tokens . they remove redundant token and then resume the reasoning generation . |
| Outcome: | The proposed method shows strong performance on reasoning-intensive benchmarks without training. |
Correct, Concise and Complete: Multi-stage Training For Adaptive Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) increase test-time computation, often in the form of chain-of-thought (CoT) however, reasoning traces can become unnecessarily long, increasing computation costs without improving accuracy and sometimes even degrading performance. |
| Approach: | They propose a multi-stage efficient reasoning method that combines supervised fine-tuning with reinforcement learning using an adaptive length penalty. |
| Outcome: | The proposed method reduces response length by an average of 28% for 8B models and 40% for 32B models while incurring only minor performance drops of 1.6 and 2.5 points, respectively. |
Too Long, Do Re-weighting for Efficient LLM Reasoning Compression (2026.acl-long)
Copied to clipboard
Zhong-Zhi Li, Xiao Liang, Zihao Tang, Lei Ji, Peijie Wang, Haotian Xu, Xing W, Haizhen Huang, Weiwei Deng, Yeyun Gong, Zhijiang Guo, Xiao Liu, Fei Yin, Cheng-Lin Liu
| Challenge: | Large Language Models (LLMs) have recently achieved remarkable progress on complex reasoning tasks by leveraging extended Chain-of-Thought (CoT) techniques. |
| Approach: | They propose a method that uses Extended Chain-of-Thought (EFT) to reduce the number of output tokens by nearly 40% while maintaining the accuracy of the reasoning. |
| Outcome: | The proposed method reduces the number of output tokens by nearly 40% while maintaining the accuracy of the reasoning. |
Tracing and Dissecting How LLMs Recall Factual Knowledge for Real World Questions (2025.acl-long)
Copied to clipboard
| Challenge: | Recent advances in large language models have shown promising ability to perform commonsense reasoning. |
| Approach: | They propose a two-dimensional analysis framework that incorporates token back-tracing and token decoding to uncover how LLMs conduct factual knowledge recall. |
| Outcome: | The proposed framework shows that LLMs lack relevant knowledge but struggle to select the most accurate information based on context during the retrieval and rerank phase. |
Can LLMs Learn From Mistakes? An Empirical Study on Reasoning Tasks (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work has shown that simple learning can enhance the chain-of-thought (CoT) reasoning of large language models. |
| Approach: | They construct mistake-correction datasets to identify and correct mistakes in CoTs . they conclude that LLMs can learn from mistakes to enhance their CoT reasoning . |
| Outcome: | The proposed datasets show that LLMs can learn from mistakes to enhance their CoT reasoning performance. |