Budget-Aware Anytime Reasoning with LLM-Synthesized Preference Data (2026.findings-acl)
Copied to clipboard
Xuanming Zhang, Shwan Ashrafi, Aziza Mirsaidova, Amir H. Rezaeian, Miguel Ballesteros, Lydia Chilton, Zhou Yu, Dan Roth
| Challenge: | Recent work has explored reasoning efficiency via test-time scaling and early exit strategies. |
| Approach: | They propose an anytime reasoning framework and the Anytime Index to improve model quality . they also propose an inference-time self-improvement method to produce better intermediate solutions . |
| Outcome: | The proposed method improves on NaturalPlan, AIME, and GPQA datasets and improves reasoning quality and efficiency under budget constraints. |
Similar Papers
Reasoning in Token Economies: Budget-Aware Evaluation of LLM Reasoning Strategies (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing evaluations that focus on performance metrics miss a key factor: increased effectiveness due to additional compute. |
| Approach: | They propose to incorporate the compute budget into evaluations to provide a more informative comparison that takes into account both performance metrics and computational cost. |
| Outcome: | The proposed framework outperforms reasoning strategies when they use comparable compute resources. |
An Empirical Study of LLM Reasoning Ability Under Strict Output Length Constraint (2025.emnlp-main)
Copied to clipboard
Yi Sun, Han Wang, Jiaqiang Li, Jiacheng Liu, Xiangyu Li, Hao Wen, Yizhen Yuan, Huiwen Zheng, Yan Liang, Yuanchun Li, Yunxin Liu
| Challenge: | Large Language Models (LLMs) are a powerful tool for test-time scaling, but they are often used under time constraints. |
| Approach: | They propose to use LLMs to make models think before answering questions . they also use self-correction and best-of-N decoding to encourage deeper thinking . |
| Outcome: | The proposed models are able to achieve higher inference accuracy with extra inference computation under time constraints. |
Reasoning Aware Self-Consistency: Leveraging Reasoning Paths for Efficient LLM Sampling (2025.naacl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) generate reasoning paths before answers, but lack a systematic approach to determine optimal number of samples or select the most faithful rationale. |
| Approach: | They propose a framework that evaluates the quality of reasoning and consistency of answers for each generated sample and uses criteria-based stopping and weighted majority voting to guide early stopping decisions and rationale selection. |
| Outcome: | The proposed framework outperforms existing methods while maintaining accuracy. |
Less is More: Improving LLM Reasoning with Minimal Test-Time Intervention (2026.acl-long)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have focused on test-time scaling to improve reasoning quality but at the cost of efficiency. |
| Approach: | They propose a training-free framework that enhances reasoning accuracy and stability with minimal overhead. |
| Outcome: | The proposed framework yields consistent gains across general, coding, and STEM tasks while remaining highly efficient. |
Self-Para-Consistency: Improving Reasoning Tasks at Low Cost for Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have shown that self-consistency decoding can improve performance for complex reasoning tasks with large language models. |
| Approach: | They propose a self-consistency decoding strategy that generates multiple paraphrases for each test question and then generates reasoning paths for the original and all the paraphrased questions based on greedy decoding. |
| Outcome: | The proposed strategy reduces the sampling number and improves performance on complex reasoning tasks. |
Let’s Sample Step by Step: Adaptive-Consistency for Efficient Reasoning and Coding with LLMs (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for improving the correctness of output from large language models generate a constant number of samples per question, but Adaptive-Consistency reduces sample budget by up to 7.9 times with an average accuracy drop of less than 0.1%. |
| Approach: | They propose a model-agnostic technique that dynamically adjusts the number of samples per question using a lightweight stopping criterion. |
| Outcome: | The proposed technique reduces sample budget by 7.9 times with an average accuracy drop of less than 0.1%. |
EpiCaR: Knowing What You Don’t Know Matters for Better Reasoning in LLMs (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to improving reasoning abilities of large language models incur a significant calibration cost. |
| Approach: | They propose an epistemic learning problem that integrates reasoning and calibration into an iterative supervised training framework. |
| Outcome: | The proposed method achieves Pareto-superiority over standard baselines in accuracy and calibration. |
Current Advances in LLM Reasoning (2026.acl-tutorials)
Copied to clipboard
| Challenge: | This tutorial examines comprehensive evaluation strategies to assess the reasoning abilities of large language models (LLMs) advanced inference time methods and post-training methods that aim to make LLMs think more like humans are discussed in this tutorial. |
| Approach: | This tutorial explores comprehensive evaluation strategies to assess the reasoning abilities of large language models (LLMs) and discusses two types of methods to improve models’ reasoning: advanced inference time methods, structured and self-improvement inference methods, and post-training methods, such as RLHF, DPO, and GRPO. |
| Outcome: | This tutorial examines evaluation strategies to assess the reasoning abilities of large language models and discusses two types of methods to improve models’ reasoning. |
Cache Saver: A Modular Framework for Efficient, Affordable, and Reproducible LLM Inference (2025.findings-emnlp)
Copied to clipboard
Nearchos Potamitis, Lars Henning Klein, Bardia Mohammadi, Chongyang Xu, Attreyee Mukherjee, Niket Tandon, Laurent Bindschaedler, Akhil Arora
| Challenge: | Inference is the major cost throughout the lifecycle of a large language model (LLM). |
| Approach: | They propose a plug-and-play, asynchronous framework that facilitates high-level inference optimizations. |
| Outcome: | The proposed framework reduces cost and CO2 by 35% across methods, tasks, and LLMs. |
Token-Budget-Aware LLM Reasoning (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to enhance reasoning capabilities of large language models incur significant overhead in token usage, leading to increased costs. |
| Approach: | They propose a token-budget-aware LLM reasoning framework that adjusts the number of reasoning tokens based on the reasoning complexity of each problem. |
| Outcome: | The proposed method reduces token costs in CoT reasoning with only a slight performance reduction. |