Papers by Junze Liu
On-Policy Self-Distillation for Efficient Diffusion Language Models with Early-Stage Calibration (2026.findings-acl)
Copied to clipboard
Huaisheng Zhu, MingYu Liu, Junze Liu, Zhen Ge, Tian Wang, Jiri Gesi, Dakuo Wang, Weiqi Zhang, Houyu Zhang, Yufan Guo, Xian Li, Bing Yin, Sujay Sanghavi
| Challenge: | Recent studies have demonstrated that masked diffusion models (MDMs) can surpass autoregressive models (ARMs) in various tasks. |
| Approach: | They propose a method to calibrate early token predictions without demonstration data by distilling an unnormalized target distribution into the original model. |
| Outcome: | Experiments on math, planning, and RLHF tasks show that COPSD improves both effectiveness and efficiency, and further enhances performance when combined with supervised fine-tuning. |