TA-GRPO-d: Trajectory-Aware GRPO for Optimizing Denoising Trajectories in Diffusion LLMs (2026.acl-long)
Copied to clipboard
| Challenge: | Existing dLLMs rely on fixed denoising schedules and cannot learn efficient unmasking orders. |
| Approach: | They propose a framework that transforms dLLM decoding into a trajectory-aware policy . it uses a confidence-gated denoising strategy that decides which tokens to unmask . |
| Outcome: | The proposed model can learn which tokens to unmask and how many to unmak per step . it can learn the output quality and efficiency of the decoding path itself . |
Similar Papers
T⋆: Progressive Block Scaling for Masked Diffusion Language Models Through Trajectory Aware Reinforcement Learning (2026.acl-short)
Copied to clipboard
| Challenge: | Autoregressive (AR) modeling via next-token prediction dominates scaling practice and deployed systems. |
| Approach: | They propose a TraceRL-based curriculum for progressive block-size scaling in masked diffusion language models. |
| Outcome: | The proposed curriculum outperforms direct large-block TraceRL on two SDAR scales and three benchmarks and retains block-size-specific non-monotone updates while improving accuracy. |
AG-GRPO: Answer-Guided GRPO for Masked Diffusion Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Recent work on large language models (LLMs) has emphasized not only final-answer accuracy but also reliability of reasoning on challenging tasks. |
| Approach: | They propose an answer-guided group-relative policy optimization for masked diffusion language models which generates text through iterative mangled token restoration. |
| Outcome: | The proposed approach improves over pretrained dLLMs and prior RL methods across mathematics, puzzle-solving, and code-generation benchmarks. |
DecoCal: Decoding with Calibration in Diffusion Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Diffusion Large Language Models (DLLMs) generate text via iterative token denoising . but decoding is challenging, with many tokens appearing predictable early . |
| Approach: | They propose a Decoding framework that performs Calibration of token-level confidence across diffusion steps and leverages the calibrated results to guide decoding decisions. |
| Outcome: | Experiments on multiple DLLMs and benchmarks show that DecoCal improves generation accuracy compared to existing strategies. |
d-TreeRPO: Towards More Reliable Policy Optimization for Diffusion Language Models (2026.acl-long)
Copied to clipboard
Leyi Pan, Shuchang Tao, Yunpeng Zhai, Zheyu Fu, Liancheng Fang, Minghua He, Lingzhe Zhang, Zhaoyang Liu, Bolin Ding, Aiwei Liu, Lijie Wen
| Challenge: | Existing RL methods suffer from reliability bottlenecks due to reward sparsity and intractable computations . d-TreeRPO provides fine-grained and verifiable step-wise reward signals . |
| Approach: | They propose a reliable reinforcement learning framework for diffusion large language models that leverages tree-structured rollouts and bottom-up advantage computation based on verifiable outcome rewards. |
| Outcome: | The proposed framework outperforms baseline models and achieves significant improvements across reasoning benchmarks. |
CreditDecoding: Accelerating Parallel Decoding in Diffusion Large Language Models with Trace Credit (2026.acl-long)
Copied to clipboard
| Challenge: | Diffusion large language models generate text through iterative denoising with bidirectional attention, enabling richer contextual dependencies. |
| Approach: | They propose a training-free parallel decoding method that fuses Trace Credit with current logits to boost the confidence of correct but underconfident tokens. |
| Outcome: | The proposed method achieves 5.48 times speedup with +0.48 accuracy on LLaDA-8B and is orthogonal to mainstream inference optimizations. |
STEP: Success-Rate-Aware Trajectory-Efficient Policy Optimization (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing GRPO-based methods allocate sampling uniformly across tasks regardless of difficulty, propagate misleading learning signals and incur high sample-collection costs. |
| Approach: | They propose a framework that allocates sampling based on per-task success rates and performs fine-grained step-level optimization. |
| Outcome: | The proposed method improves sample efficiency and training stability over existing GRPO variants and three ablation variants on OSWorld and AndroidWorld. |
Improving the Throughput of Diffusion-based Large Language Models via a Training-Free Confidence-Aware Calibration (2026.findings-acl)
Copied to clipboard
Jucheng Shen, Gaurav Sarkar, Yeonju Ro, Sharath Nittur Sridhar, Zhangyang Wang, Aditya Akella, Souvik Kundu
| Challenge: | CadLLM is a plug-and-play model-agnostic with KV caching based dLLMs. |
| Approach: | They propose a lightweight adaptive method that can control the generation block size, step size, and threshold based on the average confidence score of unmasked tokens. |
| Outcome: | The proposed method can increase throughput by up to 1.1-2.28x over the state-of-the-art model with competitive accuracy. |
LEPO: Latent Reasoning Policy Optimization for Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing latent reasoning methods that use chain of thought (CoT) are limited to selecting one discrete token at each reasoning step, which potentially induces information loss. |
| Approach: | They propose a framework that injects controllable stochasticity into latent reasoning via Gumbel-Softmax, restoring LLMs' exploratory capacity and enhancing their compatibility with Reinforcement Learning (RL). |
| Outcome: | The proposed framework preserves richer information for more comprehensive reasoning and is compatible with Reinforcement Learning (RL). |
DRA-GRPO: Your GRPO Needs to Know Diverse Reasoning Paths for Mathematical Reasoning (2026.findings-acl)
Copied to clipboard
Xiwen Chen, Wenhui Zhu, Peijie Qiu, Xuanzhao Dong, Hao Wang, Haiyu Wu, Huayu Li, Aris Sotiras, Yalin Wang, Abolfazl Razi
| Challenge: | Existing methods for group-relative policy optimization rely on scalar correctness rewards that are often non-injective with respect to semantic content. |
| Approach: | They propose a framework that calibrates the reward signal using the semantic density of sampled groups. |
| Outcome: | The proposed framework outperforms strong baselines on five math benchmarks with 7,000 samples and 55 cost. |
DiffuSpec: Unlocking Diffusion Language Models for Speculative Decoding (2026.findings-acl)
Copied to clipboard
| Challenge: | Autoregressive (AR) decoding in large language models is latency-bounded by strictly sequential token generation. |
| Approach: | They propose a diffusion-based drafter that proposes multi-token candidates and then verifies them in parallel by the target model. |
| Outcome: | The proposed drafter generates multi-token proposals in a single forward pass while remaining compatible with standard AR verifiers. |