CreditDecoding: Accelerating Parallel Decoding in Diffusion Large Language Models with Trace Credit (2026.acl-long)
Copied to clipboard
| Challenge: | Diffusion large language models generate text through iterative denoising with bidirectional attention, enabling richer contextual dependencies. |
| Approach: | They propose a training-free parallel decoding method that fuses Trace Credit with current logits to boost the confidence of correct but underconfident tokens. |
| Outcome: | The proposed method achieves 5.48 times speedup with +0.48 accuracy on LLaDA-8B and is orthogonal to mainstream inference optimizations. |
Similar Papers
DecoCal: Decoding with Calibration in Diffusion Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Diffusion Large Language Models (DLLMs) generate text via iterative token denoising . but decoding is challenging, with many tokens appearing predictable early . |
| Approach: | They propose a Decoding framework that performs Calibration of token-level confidence across diffusion steps and leverages the calibrated results to guide decoding decisions. |
| Outcome: | Experiments on multiple DLLMs and benchmarks show that DecoCal improves generation accuracy compared to existing strategies. |
Focus-dLLM: Accelerating Long-Context Diffusion LLM Inference via Confidence-Guided Context Focusing (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for estimating attention importance for tokens are ineffective . dLLMs require bidirectional attention, which limits inference efficiency . |
| Approach: | They propose a training-free attention sparsification framework for efficient long-context inference . they propose 'sink-aware pruning strategy' to accurately estimate and remove redundant computation . |
| Outcome: | The proposed approach offers 29 lossless speedup under 32K context length. |
Speculative Diffusion Decoding: Accelerating Language Generation through Diffusion (2025.naacl-long)
Copied to clipboard
Jacob K Christopher, Brian R. Bartoldson, Tal Ben-Nun, Michael Cardei, Bhavya Kailkhura, Ferdinando Fioretto
| Challenge: | Existing methods to accelerate large language model inference are limited by the reliance on incremental token generation in existing draft models. |
| Approach: | They propose an adaptation of speculative decoding which uses discrete diffusion models to generate draft sequences and allows parallelization of both the drafting and verification steps. |
| Outcome: | The proposed approach provides 7.2x speedups over standard generation processes and 1.75x speed ups over existing speculative decoding approaches. |
Beyond Fully Random Masking: Attention-Guided Denoising and Optimization for Diffusion Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for full-attention dLLMs rely on random masking strategies that overlook intrinsic token dependencies. |
| Approach: | They propose an attention-guided denoising and optimization framework that aligns training and optimization with attention-derived dependencies. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on mathematical and coding benchmarks. |
DiffuSpec: Unlocking Diffusion Language Models for Speculative Decoding (2026.findings-acl)
Copied to clipboard
| Challenge: | Autoregressive (AR) decoding in large language models is latency-bounded by strictly sequential token generation. |
| Approach: | They propose a diffusion-based drafter that proposes multi-token candidates and then verifies them in parallel by the target model. |
| Outcome: | The proposed drafter generates multi-token proposals in a single forward pass while remaining compatible with standard AR verifiers. |
SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generation (2026.findings-acl)
Copied to clipboard
Shuang Cheng, Yihan Bian, Dawei Liu, Yuhua Jiang, Yihao Liu, Linfeng Zhang, Qian Yao, Zhongbo Tian, Wenhai Wang, Qipeng Guo, Kai Chen, Biqing Qi, Bowen Zhou
| Challenge: | Autoregressive (AR) language models are a dominant paradigm in the field of parallelism and non-causal modeling. |
| Approach: | They propose a blockwise discrete diffusion model that preserves AR-compatible serving while enabling parallel intra-block generation. |
| Outcome: | The proposed model achieves theoretical speedups over 5 and wall-clock speedup of 2.3 on H200 GPUs in latency-critical regimes. |
LoPT: Lossless Parallel Tokenization Acceleration for Long Context Inference of Large Language Model (2026.acl-long)
Copied to clipboard
| Challenge: | Existing parallel tokenization methods suffer from inconsistent results due to boundary artifacts that occur after merging. |
| Approach: | They propose a Lossless Parallel Tokenization framework that ensures output identical to standard sequential tokenization. |
| Outcome: | The proposed method achieves significant speedup while guaranteeing lossless tokenization. |
Mask Tokens as Prophet: Fine-Grained Cache Eviction for Efficient dLLM Inference (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing cache eviction strategies for autoregressive language models fail to account for the role of mask tokens and specific characteristics in dLLMs. |
| Approach: | They propose a training-free cache eviction framework tailored to dLLMs that denies a fully masked sequence and allows parallel decoding at the expense of memory and computation. |
| Outcome: | The proposed framework reduces the cost of memory and cache eviction and improves efficiency by reducing allocation in intermediate layers and concentrating resources on prompt-preferring heads. |
Diffusion with Truncated Blocks: Fast and High-Quality Text Generation using Truncated Block Generation (2026.findings-acl)
Copied to clipboard
| Challenge: | Diffusion-based Large Language Models (dLLMs) generate text by iteratively denoising masked sequences. |
| Approach: | They propose a method that iteratively denoises masked sequences to reduce the model's attention dilution by token-level noise while models employing sequence-level noising exhibit a reduced effect. |
| Outcome: | The proposed method improves the performance and efficiency of Diffusion-based large language models by iterating on masked sequences. |
TA-GRPO-d: Trajectory-Aware GRPO for Optimizing Denoising Trajectories in Diffusion LLMs (2026.acl-long)
Copied to clipboard
| Challenge: | Existing dLLMs rely on fixed denoising schedules and cannot learn efficient unmasking orders. |
| Approach: | They propose a framework that transforms dLLM decoding into a trajectory-aware policy . it uses a confidence-gated denoising strategy that decides which tokens to unmask . |
| Outcome: | The proposed model can learn which tokens to unmask and how many to unmak per step . it can learn the output quality and efficiency of the decoding path itself . |