On-Policy Self-Distillation for Efficient Diffusion Language Models with Early-Stage Calibration

Challenge:	Recent studies have demonstrated that masked diffusion models (MDMs) can surpass autoregressive models (ARMs) in various tasks.
Approach:	They propose a method to calibrate early token predictions without demonstration data by distilling an unnormalized target distribution into the original model.
Outcome:	Experiments on math, planning, and RLHF tasks show that COPSD improves both effectiveness and efficiency, and further enhances performance when combined with supervised fine-tuning.

Similar Papers

Overview

Challenge

Approach

Outcome

SelFusion: Self-distillation for Diffusion Language Models (2026.acl-long)

Copied to clipboard

Hyeongsoo Lim, Jinyoung Kim, Eun Seo Seo, Min Ho Jang, Ji Won Yoon

Language Models

Challenge:	Existing knowledge distillation methods for autoregressive large language models (LLMs) are not effective for reducing generation quality, but they can be useful for real-time applications.
Approach:	They propose a self-distillation framework that allows for effective KD without external teacher . they propose to use two modes of knowledge distillation to determine distillation direction .
Outcome:	The proposed framework outperforms existing methods with external teachers on instruction-following tasks.

DecoCal: Decoding with Calibration in Diffusion Large Language Models (2026.acl-long)

Copied to clipboard

Fan Xu, Huixuan Zhang, Xiaojun Wan

Language Models

Challenge:	Diffusion Large Language Models (DLLMs) generate text via iterative token denoising . but decoding is challenging, with many tokens appearing predictable early .
Approach:	They propose a Decoding framework that performs Calibration of token-level confidence across diffusion steps and leverages the calibrated results to guide decoding decisions.
Outcome:	Experiments on multiple DLLMs and benchmarks show that DecoCal improves generation accuracy compared to existing strategies.

Fast and Effective On-Policy Distillation from Reasoning Prefixes (2026.findings-acl)

Copied to clipboard

Dongxu Zhang, Zhichao Yang, Sepehr Janghorbani, Jun Han, Andrew Ressler II, Qian Qian, Gregory D Lyng, Sanjit Singh Batra, Robert E. Tillman

Language Models

Challenge:	On-policy distillation (OPD) requires expensive on-the-fly sampling of the student policy during training, which substantially increases training cost.
Approach:	They propose to use on-policy distillation to sample trajectories from student model . they propose to terminate the sampling early during distillation .
Outcome:	The proposed method matches the performance of full OPD in long reasoning outputs while reducing training FLOP by 2x–40x.

Differentially Private Learning Needs Better Model Initialization and Self-Distillation (2025.naacl-long)

Copied to clipboard

Ivoline C. Ngong, Joseph Near, Niloofar Mireshghallah

Responsible & Trustworthy NLP

Challenge:	Differentially private SGD (DPSGD) enables privacy-preserving training of language models, but often reduces utility, diversity, and linguistic quality.
Approach:	They propose a method that initializes a model using data synthesis from a small pre-trained LM with rigorous filtering, applies DP finetuning on private data, and performs self-distillation to refine outputs.
Outcome:	The proposed method outperforms vanilla DPSGD with significant improvements in lexical diversity and grammar errors.

Fast-Decoding Diffusion Language Models via Progress-Aware Confidence Schedules (2026.findings-acl)

Copied to clipboard

Amr Mohamed, Yang Zhang, Michalis Vazirgiannis, Guokan Shang

Language Models

Challenge:	SchED is a training-free, model-agnostic early-exit algorithm that terminates diffusion decoding using a progress-aware confidence threshold.
Approach:	They propose a training-free, model-agnostic early-exit algorithm that terminates diffusion decoding using a progress-aware confidence threshold.
Outcome:	The proposed algorithm achieves 4 speedups on instruction-tuned models while maintaining baseline performance on average.

d-TreeRPO: Towards More Reliable Policy Optimization for Diffusion Language Models (2026.acl-long)

Copied to clipboard

Leyi Pan, Shuchang Tao, Yunpeng Zhai, Zheyu Fu, Liancheng Fang, Minghua He, Lingzhe Zhang, Zhaoyang Liu, Bolin Ding, Aiwei Liu, Lijie Wen

Language Models

Challenge:	Existing RL methods suffer from reliability bottlenecks due to reward sparsity and intractable computations . d-TreeRPO provides fine-grained and verifiable step-wise reward signals .
Approach:	They propose a reliable reinforcement learning framework for diffusion large language models that leverages tree-structured rollouts and bottom-up advantage computation based on verifiable outcome rewards.
Outcome:	The proposed framework outperforms baseline models and achieves significant improvements across reasoning benchmarks.

Improving the Throughput of Diffusion-based Large Language Models via a Training-Free Confidence-Aware Calibration (2026.findings-acl)

Copied to clipboard

Jucheng Shen, Gaurav Sarkar, Yeonju Ro, Sharath Nittur Sridhar, Zhangyang Wang, Aditya Akella, Souvik Kundu

Semantic Text Processing

Challenge:	CadLLM is a plug-and-play model-agnostic with KV caching based dLLMs.
Approach:	They propose a lightweight adaptive method that can control the generation block size, step size, and threshold based on the average confidence score of unmasked tokens.
Outcome:	The proposed method can increase throughput by up to 1.1-2.28x over the state-of-the-art model with competitive accuracy.

Reinforcement Learning for Diffusion LLMs via Energy-Based Gibbs Alignment (2026.acl-long)

Copied to clipboard

Yijia Fan, Jing Yang, Mingyu Liu, Kaitong Cai, Jian Wang, Keze Wang, Jusheng Zhang

Reasoning

Challenge:	Diffusion Large Language Models (dLLMs) offer parallel decoding and bidirectional context modeling . aligning dLLms with reinforcement learning (RL) remains a challenge .
Approach:	They propose a variational framework that reformulates RL for dLLMs as a distribution matching problem.
Outcome:	The proposed framework reformulates RL for dLLMs as a distribution matching problem.

DiffuSpec: Unlocking Diffusion Language Models for Speculative Decoding (2026.findings-acl)

Copied to clipboard

Guanghao Li, Zhihui Fu, Min Fang, Qibin Zhao, Ming Tang, Chun Yuan, Jun Wang

Language Models

Challenge:	Autoregressive (AR) decoding in large language models is latency-bounded by strictly sequential token generation.
Approach:	They propose a diffusion-based drafter that proposes multi-token candidates and then verifies them in parallel by the target model.
Outcome:	The proposed drafter generates multi-token proposals in a single forward pass while remaining compatible with standard AR verifiers.

FIRST: Teach A Reliable Large Language Model Through Efficient Trustworthy Distillation (2024.emnlp-main)

Copied to clipboard

KaShun Shum, Minrui Xu, Jianshu Zhang, Zixin Chen, Shizhe Diao, Hanze Dong, Jipeng Zhang, Muhammad Raza

Responsible & Trustworthy NLP

Challenge:	Experimental results show that a well-calibrated model is more reliable than a fine-tuned model due to “tuning-induced mis-calibration”.
Approach:	They propose a method which utilizes a small portion of teacher’s knowledge to obtain a reliable language model in a cost-efficient way.
Outcome:	The proposed method reduces the computational burden by utilizing teacher's knowledge to obtain a reliable language model in a cost-efficient way.