Offloaded Reasoning: Efficient Inference for Large Language Models via Modular Reasoning and Refinement (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) demonstrate strong reasoning capabilities but are expensive to run at inference time, limiting their practical deployment. |
| Approach: | They propose Offloaded Reasoning, a modular strategy where a lightweight model generates intermediate reasoning traces that are then used by a larger model to produce the final answer. |
| Outcome: | The proposed approach achieves faster inferences than full large-model reasoning with minimal accuracy loss while recovering or exceeding full accuracy at substantially lower cost. |
Similar Papers
Out-of-Context Reasoning in Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a lightweight technique trains only new token embeddings on axioms and evaluates them on unseen tasks. |
| Approach: | They propose a lightweight technique that trains only new token embeddings on axioms . they train only new embeddables and evaluate them on unseen tasks . |
| Outcome: | The proposed technique trains only new token embeddings on axioms and evaluates them on unseen tasks. |
Tandem: Riding Together with Large and Small Language Models for Efficient Reasoning (2026.findings-acl)
Copied to clipboard
Zichuan Fu, Xian Wu, Guojing Li, Yejing Wang, Yijun Chen, Zhao Zihao, Luo Yixuan, Hanyu Yan, Yefeng Zheng, Xiangyu Zhao
| Challenge: | Recent advances in large language models (LLMs) have catalyzed the rise of reasoningintensive inference paradigms, where models perform explicit step-by-step reasoning before generating final answers. |
| Approach: | They propose a large-small LLM collaboration framework that synergizes large and small language models to achieve high-quality reasoning with significantly reduced computational cost. |
| Outcome: | The proposed framework outperforms the mentor LLM while preserving the benefits of the thinking paradigm of LLMs. |
Current Advances in LLM Reasoning (2026.acl-tutorials)
Copied to clipboard
| Challenge: | This tutorial examines comprehensive evaluation strategies to assess the reasoning abilities of large language models (LLMs) advanced inference time methods and post-training methods that aim to make LLMs think more like humans are discussed in this tutorial. |
| Approach: | This tutorial explores comprehensive evaluation strategies to assess the reasoning abilities of large language models (LLMs) and discusses two types of methods to improve models’ reasoning: advanced inference time methods, structured and self-improvement inference methods, and post-training methods, such as RLHF, DPO, and GRPO. |
| Outcome: | This tutorial examines evaluation strategies to assess the reasoning abilities of large language models and discusses two types of methods to improve models’ reasoning. |
Can Small Language Models Help Large Language Models Reason Better?: LM-Guided Chain-of-Thought (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing frameworks for guiding a language model in reasoning tasks are limited by their tendency to generate low-quality rationales that are repetitive and vacuous. |
| Approach: | They propose a framework that leverages a lightweight language model for guiding a black-box large LM in reasoning tasks. |
| Outcome: | The proposed framework outperforms baselines in answer prediction accuracy. |
GSM-Noise: Exploring and Enhancing Large Language Models’ Reasoning under Noisy Inputs (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models struggle when dealing with complex, ill-formed, or noisy inputs . open-source models are less robust, while closed-source ones are more robust . |
| Approach: | They propose to use GSM-Noise to refine inputs before engaging in in-depth analysis to improve LLM robustness under noisy conditions. |
| Outcome: | The proposed model can achieve consistent performance gains under noisy conditions with prompt engineering, supervised finetuning, and reinforcement learning. |
Towards Reasoning in Large Language Models: A Survey (2023.findings-acl)
Copied to clipboard
| Challenge: | Reasoning is a fundamental aspect of human intelligence that plays a crucial role in many intellectual activities. |
| Approach: | They propose to improve LLMs' ability to elicit reasoning by providing exemplars or prompts to model reasoning. |
| Outcome: | This paper provides a comprehensive overview of the state of knowledge on reasoning in large language models. |
Efficient Inference for Large Language Models –Algorithm, Model, and System (2025.emnlp-tutorials)
Copied to clipboard
| Challenge: | Inference of LLMs incurs high computational costs, memory access overhead, and memory usage, leading to inefficiencies in terms of latency, throughput, power consumption, and storage. |
| Approach: | This tutorial introduces the basics of efficient inference for LLMs and explains how to diagnose efficiency bottlenecks for a given workload on specific hardware. |
| Outcome: | The tutorial introduces the basic concepts of modern LLMs, software and hardware. |
Over-Reasoning and Redundant Calculation of Large Language Models (2024.eacl-short)
Copied to clipboard
| Challenge: | Large language models (LLMs) can solve problems step-by-step, but it is unclear whether they know when to use CoT and whether they are always necessary. |
| Approach: | They propose to use LLMs to generate redundant calculations and reasoning on a manually constructed math QA dataset, GSM8K-Zero. |
| Outcome: | The proposed model generates redundant calculations and reasoning on a manually constructed math QA dataset, but it is unclear whether it is necessary to use CoT reasoning. |
LLM2: Let Large Language Models Harness System 2 Reasoning (2025.naacl-short)
Copied to clipboard
| Challenge: | Empirical results on mathematical reasoning benchmarks substantiate the efficacy of Large language models (LLMs). |
| Approach: | They propose a framework that combines an LLM with a process-based verifier to generate plausible candidates and provide timely process-driven feedback to distinguish desirable and undesirable outputs. |
| Outcome: | Empirical results show that LLM2 improves accuracy on GSM8K and self-consistency increases major@20 accuracy. |
Correct, Concise and Complete: Multi-stage Training For Adaptive Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) increase test-time computation, often in the form of chain-of-thought (CoT) however, reasoning traces can become unnecessarily long, increasing computation costs without improving accuracy and sometimes even degrading performance. |
| Approach: | They propose a multi-stage efficient reasoning method that combines supervised fine-tuning with reinforcement learning using an adaptive length penalty. |
| Outcome: | The proposed method reduces response length by an average of 28% for 8B models and 40% for 32B models while incurring only minor performance drops of 1.6 and 2.5 points, respectively. |