Improving the Throughput of Diffusion-based Large Language Models via a Training-Free Confidence-Aware Calibration (2026.findings-acl)
Copied to clipboard
Jucheng Shen, Gaurav Sarkar, Yeonju Ro, Sharath Nittur Sridhar, Zhangyang Wang, Aditya Akella, Souvik Kundu
| Challenge: | CadLLM is a plug-and-play model-agnostic with KV caching based dLLMs. |
| Approach: | They propose a lightweight adaptive method that can control the generation block size, step size, and threshold based on the average confidence score of unmasked tokens. |
| Outcome: | The proposed method can increase throughput by up to 1.1-2.28x over the state-of-the-art model with competitive accuracy. |
Similar Papers
Focus-dLLM: Accelerating Long-Context Diffusion LLM Inference via Confidence-Guided Context Focusing (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for estimating attention importance for tokens are ineffective . dLLMs require bidirectional attention, which limits inference efficiency . |
| Approach: | They propose a training-free attention sparsification framework for efficient long-context inference . they propose 'sink-aware pruning strategy' to accurately estimate and remove redundant computation . |
| Outcome: | The proposed approach offers 29 lossless speedup under 32K context length. |
DecoCal: Decoding with Calibration in Diffusion Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Diffusion Large Language Models (DLLMs) generate text via iterative token denoising . but decoding is challenging, with many tokens appearing predictable early . |
| Approach: | They propose a Decoding framework that performs Calibration of token-level confidence across diffusion steps and leverages the calibrated results to guide decoding decisions. |
| Outcome: | Experiments on multiple DLLMs and benchmarks show that DecoCal improves generation accuracy compared to existing strategies. |
UniSpec: Training-Free Speculative Decoding for Robust LLM Acceleration Across Languages and Hardware (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for speculative decoding ignore device-specific verification costs and lack of mechanisms to assess draft token quality. |
| Approach: | They propose a training-free, lossless speculative decoding framework that enables robust, plug-and-play LLM acceleration across diverse hardware configurations and languages. |
| Outcome: | The proposed framework outperforms existing training-free methods while maintaining identical output quality across different hardware environments. |
StitchLLM: Serving LLMs, One Block at a Time (2025.acl-long)
Copied to clipboard
Bodun Hu, Shuozhe Li, Saurabh Agarwal, Myungjin Lee, Akshay Jajoo, Jiamin Li, Le Xu, Geon-Woo Kim, Donghyun Kim, Hong Xu, Amy Zhang, Aditya Akella
| Challenge: | Existing techniques like distillation and pruning are not efficient for large language models. |
| Approach: | They propose a dynamic model routing framework that uses a powerful bottom model to process all queries and a lightweight routing mechanism to allocate computational resources appropriately. |
| Outcome: | The proposed framework improves system throughput while minimizing performance degradation. |
Fast-Decoding Diffusion Language Models via Progress-Aware Confidence Schedules (2026.findings-acl)
Copied to clipboard
| Challenge: | *SchED* is a training-free, model-agnostic early-exit algorithm that terminates diffusion decoding using a progress-aware confidence threshold. |
| Approach: | They propose a training-free, model-agnostic early-exit algorithm that terminates diffusion decoding using a progress-aware confidence threshold. |
| Outcome: | The proposed algorithm achieves 4 speedups on instruction-tuned models while maintaining baseline performance on average. |
dLLM: Simple Diffusion Language Modeling (2026.acl-demo)
Copied to clipboard
| Challenge: | diffusion language models (DLMs) are evolving rapidly but many lack transparent implementations or are scattered across codebases. |
| Approach: | They propose an open-source framework that unifies diffusion language modeling components while remaining flexible enough to support new methods and architectures. |
| Outcome: | dLLM unifies the core components of diffusion language modeling and makes them easy to customize for new designs. |
Towards Efficient and Effective Diffusion Language Model Inference via Semantic-Aware Adaptive Denoising (2026.acl-long)
Copied to clipboard
| Challenge: | Existing acceleration works cannot accurately detect semantically stabilized tokens and then skip computation, leading to sub-optimal speedup in practice. |
| Approach: | They propose a semantic-aware adaptive denoising framework that encodes scalar confidence scores into an evolution-awful feature vector and clusters vectors proactively and adaptively identify semantically converged tokens. |
| Outcome: | The proposed framework outperforms the SOTA competitor in speed and quality . it can detect semantically stabilized tokens and skip computation, resulting in sub-optimal speedup . |
DiffuSpec: Unlocking Diffusion Language Models for Speculative Decoding (2026.findings-acl)
Copied to clipboard
| Challenge: | Autoregressive (AR) decoding in large language models is latency-bounded by strictly sequential token generation. |
| Approach: | They propose a diffusion-based drafter that proposes multi-token candidates and then verifies them in parallel by the target model. |
| Outcome: | The proposed drafter generates multi-token proposals in a single forward pass while remaining compatible with standard AR verifiers. |
Hardware-Aware Parallel Prompt Decoding for Memory-Efficient Acceleration of LLM Inference (2025.findings-emnlp)
Copied to clipboard
Hao Mark Chen, Wayne Luk, Yiu Ka Fai Cedric, Rui Li, Konstantin Mishchenko, Stylianos Venieris, Hongxiang Fan
| Challenge: | Auto-regressive decoding of Large Language Models results in significant overheads in hardware performance . a novel parallel prompt decoding approach is proposed to overcome these limitations . |
| Approach: | They propose a parallel prompt decoding that uses a single model for speculation and verification. |
| Outcome: | The proposed approach speeds up auto-regressive decoding of large language models 2.49 times . it can be used on mobileLlama to Vicuna-13B on a wide range of benchmarks . |
TrimLLM: Progressive Layer Dropping for Domain-Specific LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | specialized language models do not show simultaneous memory saving and inference speedup at deployment time. |
| Approach: | They develop a layer-wise specialization technique that reduces the depth of LLMs by progressive layer dropping and compares it to other algorithms for inference. |
| Outcome: | The proposed model retains LLMs’ capacity in specific domains and achieves inference speedup irrespective of hardware and deep learning frameworks. |