Scaling LLM Inference Efficiently with Optimized Sample Compute Allocation (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing methods to optimize sample allocations for large language models fail to account for the optimal sampling configuration. |
| Approach: | They propose an algorithm that optimizes sample allocation by finding an optimal mix of different inference configurations. |
| Outcome: | The proposed algorithm achieves better accuracy on SWE-Bench with 3x less compute than the default configuration. |
Similar Papers
When Life Gives You Samples: The Benefits of Scaling up Inference Compute for Multilingual LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in large language models have shifted focus toward scaling inference-time compute. |
| Approach: | They propose to scale inference-time compute in a multilingual, multi-task setting . they propose to use m-ArenaHard-v2.0 prompts to sample multiple outputs in parallel . |
| Outcome: | The proposed solutions achieve an average +6.8 jump in win-rates for 8B models on m-ArenaHard-v2.0 prompts in non-English languages against proprietary models like Gemini. |
Enhancing Efficiency and Exploration in Reinforcement Learning for LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches allocate an equal number of rollouts to all questions during the RL process, which is inefficient. |
| Approach: | They propose a mechanism for dynamically allocating rollout budgets based on the difficulty of the problems, enabling more efficient RL training. |
| Outcome: | The proposed model improves response precision while preserving exploratory ability to uncover potential correct pathways. |
Efficient Inference for Large Language Models –Algorithm, Model, and System (2025.emnlp-tutorials)
Copied to clipboard
| Challenge: | Inference of LLMs incurs high computational costs, memory access overhead, and memory usage, leading to inefficiencies in terms of latency, throughput, power consumption, and storage. |
| Approach: | This tutorial introduces the basics of efficient inference for LLMs and explains how to diagnose efficiency bottlenecks for a given workload on specific hardware. |
| Outcome: | The tutorial introduces the basic concepts of modern LLMs, software and hardware. |
Let’s Sample Step by Step: Adaptive-Consistency for Efficient Reasoning and Coding with LLMs (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for improving the correctness of output from large language models generate a constant number of samples per question, but Adaptive-Consistency reduces sample budget by up to 7.9 times with an average accuracy drop of less than 0.1%. |
| Approach: | They propose a model-agnostic technique that dynamically adjusts the number of samples per question using a lightweight stopping criterion. |
| Outcome: | The proposed technique reduces sample budget by 7.9 times with an average accuracy drop of less than 0.1%. |
SPaCe: Unlocking Sample-Efficient Large Language Models Training With Self-Pace Curriculum Learning (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing training pipelines sample training examples uniformly across steps or epochs, ignoring differences in difficulty, redundancy, and learning value, which slows learning and wastes computation. |
| Approach: | They propose a self-paced learning framework that enables efficient learning based on the capability of the model being trained through optimizing which data to use and when. |
| Outcome: | The proposed framework achieves comparable or better accuracy than state-of-the-art baselines while using up to (100 times) fewer samples. |
ConMA : Confidence-Guided Kernel Sampling with Multi-Stage Aggregation for LLM Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to test-time scaling rely on external verifiers and one-shot independent sampling. |
| Approach: | They propose a test-time scaling framework that reallocates a fixed inference budget into iterative sample–filter–diversify–select cycles. |
| Outcome: | ConMA outperforms baselines on multiple benchmarks while converging early with only 18 samples on average, substantially reducing inference cost. |
From Data-Centric to Sample-Centric: Enhancing LLM Reasoning via Progressive Optimization (2026.acl-long)
Copied to clipboard
| Challenge: | Reinforcement learning with verifiable rewards (RLVR) has recently advanced the reasoning capabilities of large language models (LLMs). |
| Approach: | They propose a method that incorporates partial solution prefixes from expert demonstrations to guide the policy. |
| Outcome: | The proposed methods outperform strong baselines, yielding faster convergence and a higher performance ceiling. |
The Best of Both Worlds: Combining Parallel and Sequential Inference Scaling via Aggregation Fine-Tuning (2026.findings-acl)
Copied to clipboard
| Challenge: | Empirical results show that AFT-trained models achieve substantial gains with test-time scaling. |
| Approach: | They introduce a supervised fine-tuning paradigm where models synthesize multiple draft responses into a single, refined answer. |
| Outcome: | Empirical results show that AFT-trained models outperform baseline models while eliminating external guidance. |
Outlier-weighed Layerwise Sampling for LLM Fine-tuning (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are a powerful tool for processing complex natural language processing tasks. |
| Approach: | They propose an approach to fine-tune LLMs with outliers and a gradient low-rank projection to increase the number of fine-sampled layers without a proportional increase in memory costs. |
| Outcome: | The proposed approach outperforms baseline approaches while being more memory efficient. |
Exploring Hybrid Sampling Inference for Aspect-based Sentiment Analysis (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for inference require multiple sampling with preset size . however, it is a high-cost method that requires multiple sampling . |
| Approach: | They propose a method that combines multiple and single sampling to greatly reduce the cost of multiple sampling without sacrificing performance. |
| Outcome: | The proposed method greatly reduces the cost of multiple sampling without sacrificing performance. |