LanguageFlow: Advancing Diffusion Language Generation with Probabilistic Flows (2024.naacl-long)
Copied to clipboard
| Challenge: | Recent work has demonstrated success in controlling sentence attributes and structure based on diffusion language models. |
| Approach: | They propose a language-rectified flow method that reformulates standard probabilistic flow models to learn ordinary differential equations to transport between the source and target distributions. |
| Outcome: | The proposed method outperforms baselines on three fine-grained control tasks and multiple high-quality text editing tasks. |
Similar Papers
Generating Text from Language Models (2023.acl-tutorials)
Copied to clipboard
| Challenge: | a growing percentage of natural language processing tasks focus on the generation of text from probabilistic language models. |
| Approach: | They will provide a centralized discussion of critical considerations when choosing how to generate from a language model. |
| Outcome: | This tutorial will provide a centralized discussion of critical considerations when choosing how to generate from a language model. |
Segment-Level Diffusion: A Framework for Controllable Long-Form Generation with Diffusion Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Diffusion models have shown promise in text generation, but often struggle with generating long, coherent, and contextually accurate text. |
| Approach: | They propose a framework that enhances diffusion-based text generation through text segmentation, robust representation training with adversarial and contrastive learning, and improved latent-space guidance. |
| Outcome: | The proposed framework improves diffusion-based text generation and improves scalability and fluency. |
CodeFusion: A Pre-trained Diffusion Model for Code Generation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for code generation from natural language do not allow reconsidering earlier tokens . prior work has explored grouped beam search or nucleus sampling to generate diverse text. |
| Approach: | They propose a diffusion code generation model that iteratively denoises a program conditioned on the encoded natural language. |
| Outcome: | The proposed model outperforms state-of-the-art models in accuracy and diversity compared to existing models. |
dLLM: Simple Diffusion Language Modeling (2026.acl-demo)
Copied to clipboard
| Challenge: | diffusion language models (DLMs) are evolving rapidly but many lack transparent implementations or are scattered across codebases. |
| Approach: | They propose an open-source framework that unifies diffusion language modeling components while remaining flexible enough to support new methods and architectures. |
| Outcome: | dLLM unifies the core components of diffusion language modeling and makes them easy to customize for new designs. |
One-step Nonautoregressive Natural Language Generation with Shortcut Flow Matching Models (2026.acl-short)
Copied to clipboard
| Challenge: | Recent diffusion-based approaches to text generation are inefficient due to the need for multiple denoising steps. |
| Approach: | They propose a shortcut flow-matching model that learns to directly predict multi-step denoising outcomes in a single step. |
| Outcome: | The proposed model improves on three datasets and can predict multi-step denoising outcomes in a single step. |
Revisiting Simple Neural Probabilistic Language Models (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent advances in language modeling have been driven not only by advances in neural architectures, but also through hardware and optimization improvements. |
| Approach: | They revisit the neural probabilistic language model (NPLM) of Bengio et al. (2003) which simply concatenates word embeddings within a fixed window and passes the result through a feed-forward network to predict the next word. |
| Outcome: | The proposed model performs better on word-level language model benchmarks than a baseline Transformer with short input contexts but struggles to handle long-term dependencies. |
Speculative Diffusion Decoding: Accelerating Language Generation through Diffusion (2025.naacl-long)
Copied to clipboard
Jacob K Christopher, Brian R. Bartoldson, Tal Ben-Nun, Michael Cardei, Bhavya Kailkhura, Ferdinando Fioretto
| Challenge: | Existing methods to accelerate large language model inference are limited by the reliance on incremental token generation in existing draft models. |
| Approach: | They propose an adaptation of speculative decoding which uses discrete diffusion models to generate draft sequences and allows parallelization of both the drafting and verification steps. |
| Outcome: | The proposed approach provides 7.2x speedups over standard generation processes and 1.75x speed ups over existing speculative decoding approaches. |
Advancing Reasoning in Diffusion Language Models with Denoising Process Rewards (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for improving reasoning in diffusion language models rely on outcome-based rewards that provide no direct supervision over the denoising process. |
| Approach: | They propose a method that provides a process-level reinforcement signal over denoising trajectory of diffusion language models. |
| Outcome: | Experiments on challenging reasoning benchmarks show that the proposed model improves reasoning stability, interpretability and overall performance. |
DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models (2023.acl-long)
Copied to clipboard
| Challenge: | Existing generative masked language models have a shared training objective, i.e., denoising. |
| Approach: | They propose a noise schedule for the forward diffusion process that controls the degree of noise added at each step based on the information of each token. |
| Outcome: | The proposed model improves on existing models in terms of perplexity and BLEU score. |
Continuous Language Generative Flow (2021.acl-long)
Copied to clipboard
| Challenge: | Recent years have witnessed various types of generative models for natural language generation (NLG), especially RNNs or transformers. |
| Approach: | They propose a flow-based language generation model that adapts flow-derived generative models to language generation via continuous input embeddings, adapted affine coupling structures, and a novel architecture for autoregressive text generation. |
| Outcome: | The proposed model improves on QG and NMT and improves performance over baselines on SQuAD and TVQA and NML16. |