Efficient Inference for Large Language Models –Algorithm, Model, and System (2025.emnlp-tutorials)
Copied to clipboard
| Challenge: | Inference of LLMs incurs high computational costs, memory access overhead, and memory usage, leading to inefficiencies in terms of latency, throughput, power consumption, and storage. |
| Approach: | This tutorial introduces the basics of efficient inference for LLMs and explains how to diagnose efficiency bottlenecks for a given workload on specific hardware. |
| Outcome: | The tutorial introduces the basic concepts of modern LLMs, software and hardware. |
Similar Papers
Current Advances in LLM Reasoning (2026.acl-tutorials)
Copied to clipboard
| Challenge: | This tutorial examines comprehensive evaluation strategies to assess the reasoning abilities of large language models (LLMs) advanced inference time methods and post-training methods that aim to make LLMs think more like humans are discussed in this tutorial. |
| Approach: | This tutorial explores comprehensive evaluation strategies to assess the reasoning abilities of large language models (LLMs) and discusses two types of methods to improve models’ reasoning: advanced inference time methods, structured and self-improvement inference methods, and post-training methods, such as RLHF, DPO, and GRPO. |
| Outcome: | This tutorial examines evaluation strategies to assess the reasoning abilities of large language models and discusses two types of methods to improve models’ reasoning. |
Energy Considerations of Large Language Model Inference and Efficiency Optimizations (2025.acl-long)
Copied to clipboard
| Challenge: | Prior benchmarking efforts focused on latency reduction in idealized settings, often overlooking real-world inference workloads that shape energy use. |
| Approach: | They propose a modeling approach that approximates real-world LLM workflows . they show that the effectiveness of inference optimizations is sensitive to workload geometry . |
| Outcome: | The proposed approach reduces energy use by 73% from unoptimized baselines. |
LLM in a flash: Efficient Large Language Model Inference with Limited Memory (2024.acl-long)
Copied to clipboard
Keivan Alizadeh, Seyed Iman Mirzadeh, Dmitry Belenko, S. Khatamifard, Minsik Cho, Carlo C Del Mundo, Mohammad Rastegari, Mehrdad Farajtabar
| Challenge: | Large language models (LLMs) have high computational and memory requirements, especially for devices with limited memory. |
| Approach: | They propose a method that stores model parameters in flash memory but brings them on demand to DRAM . authors propose two techniques to optimize for reading data in larger, more contiguous chunks . |
| Outcome: | The proposed method reduces the volume of data transferred from flash and reads data in larger, more contiguous chunks. |
Towards Sustainable NLP: Insights from Benchmarking Inference Energy in Large Language Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) are recognized for their exceptional generative capabilities and versatility across various tasks. |
| Approach: | They conduct a comprehensive benchmarking of LLM inference energy across a wide range of NLP tasks to determine the impact of different models, tasks, prompts, and system-related factors on inference. |
| Outcome: | The proposed model energy benchmarks show that quantization and optimal batch sizes can significantly reduce energy usage. |
Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding (2024.findings-acl)
Copied to clipboard
Heming Xia, Zhe Yang, Qingxiu Dong, Peiyi Wang, Yongqi Li, Tao Ge, Tianyu Liu, Wenjie Li, Zhifang Sui
| Challenge: | Large Language Models (LLMs) have a high inference latency stemming from autoregressive decoding. |
| Approach: | They propose a novel decoding paradigm that drafts multiple tokens and verifies them in parallel . they aim to provide a catalyst for further research on Speculative Decoding . |
| Outcome: | The proposed method drafts multiple tokens and verifies them in parallel . it can be used to accelerate inference in large language models. |
Towards Reasoning in Large Language Models: A Survey (2023.findings-acl)
Copied to clipboard
| Challenge: | Reasoning is a fundamental aspect of human intelligence that plays a crucial role in many intellectual activities. |
| Approach: | They propose to improve LLMs' ability to elicit reasoning by providing exemplars or prompts to model reasoning. |
| Outcome: | This paper provides a comprehensive overview of the state of knowledge on reasoning in large language models. |
Towards Effective and Efficient Multi-Agent Language Model Systems: Foundations, Prospects, and Applications (2026.acl-tutorials)
Copied to clipboard
| Challenge: | Multi-agent systems powered by large language models still face challenges . tutorial focuses on three core components to build effective and efficient systems . |
| Approach: | This tutorial introduces recent advances in building effective and efficient multi-agent LLM systems . it focuses on three core components: model distillation, dynamic routing, memory- and compute efficient serving . |
| Outcome: | This tutorial introduces state-of-the-art techniques for building efficient and efficient multi-agent LLM systems . it covers coordination and communication among agents, crucial for collective performance . |
Adaptation of Large Language Models (2025.naacl-tutorial)
Copied to clipboard
| Challenge: | a tutorial on adaptation of large language models addresses the growing demand for models that go beyond static capabilities. |
| Approach: | This tutorial will provide an overview of dynamic, domain-specific, and task-adaptive LLM adaptation techniques. |
| Outcome: | This tutorial will outline dynamic, domain-specific, and task-adaptive LLM adaptation techniques. |
The Impact of Inference Acceleration on Bias of LLMs (2025.naacl-long)
Copied to clipboard
| Challenge: | Recent work suggests strategies to increase inference efficiency with LLMs . however, these strategies may inadvertently lead to some side-effects. |
| Approach: | They propose to optimize inference acceleration strategies such as quantization, pruning, and caching to reduce inference cost and latency while maintaining predictive performance. |
| Outcome: | The proposed strategies reduce cost and latency while maintaining predictive performance while preserving the model size. |
A Survey of Confidence Estimation and Calibration in Large Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated impressive capabilities across a wide range of tasks in various domains, but they can be unreliable due to factual errors in their generations. |
| Approach: | They summarize recent advances in LLM confidence estimation and calibration and outline their main lessons learned. |
| Outcome: | The proposed methods can be used to assess the reliability of models and to calibrate them across tasks. |