InfoDiffusion: Information Entropy Aware Diffusion Process for Non-Autoregressive Text Generation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing text diffusion models have failed to capture the difference between the “easy-first” process and the “keyword-first" process of humans. |
| Approach: | They propose a non-autoregressive text diffusion model that incorporates a "keyinfo-first" generation strategy and a noise schedule based on the amount of text information. |
| Outcome: | The proposed model outperforms the baseline model in terms of generation quality and diversity, and higher sampling efficiency. |
Similar Papers
TESS: Text-to-Text Self-Conditioned Simplex Diffusion (2024.eacl-long)
Copied to clipboard
Rabeeh Karimi Mahabadi, Hamish Ivison, Jaesung Tae, James Henderson, Iz Beltagy, Matthew Peters, Arman Cohan
| Challenge: | Existing models for diffusion generation are expensive and discrete, resulting in a large number of diffusion steps to generate text. |
| Approach: | They propose a text diffusion model that is fully non-autoregressive and employs a new form of self-conditioning and applies the diffusion process on the logit simplex space rather than the learned embedding space. |
| Outcome: | The proposed model outperforms state-of-the-art non-autoregressive models, requires fewer diffusion steps with minimal drop in performance, and is competitive with pretrained autoregressive sequence-to-sequence models. |
Unifying Continuous and Discrete Text Diffusion with Non-simultaneous Diffusion Processes (2025.acl-long)
Copied to clipboard
| Challenge: | Experimental results demonstrate NeoDiff’s superior performance compared to baselines of non-autoregressive continuous and discrete diffusion models, iterative-based methods and autoregressive diffusion-based approaches. |
| Approach: | They propose a discrete and continuous diffusion model that integrates the strengths of discrete, continuous and continuous approaches. |
| Outcome: | The proposed model unifies the theories of discrete and continuous diffusion models, offering a more principled and effective framework for text generation. |
Diffusion-NAT: Self-Prompting Discrete Diffusion for Non-Autoregressive Text Generation (2024.eacl-long)
Copied to clipboard
| Challenge: | Existing non-autoregressive (NAR) text-to-text generation methods are unable to generate coherent and fluent texts due to discrete nature of text. |
| Approach: | They propose to integrate discrete diffusion models (DDM) into NAR text-to-text generation and integrate BART to improve the performance. |
| Outcome: | The proposed method outperforms competing methods and surpasses autoregressive methods on 7 datasets. |
Text Diffusion Model with Encoder-Decoder Transformers for Sequence-to-Sequence Generation (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing diffusion models are applied to continuous feature space while texts are sequences of discrete categorical tokens. |
| Approach: | They propose to use an encoder-decoder Transformer architecture to approach sequence-to-sequence text generation. |
| Outcome: | The proposed model improves on five sequence-to-sequence generation tasks compared to other diffusion-based models regarding text quality and inference time. |
One-step Nonautoregressive Natural Language Generation with Shortcut Flow Matching Models (2026.acl-short)
Copied to clipboard
| Challenge: | Recent diffusion-based approaches to text generation are inefficient due to the need for multiple denoising steps. |
| Approach: | They propose a shortcut flow-matching model that learns to directly predict multi-step denoising outcomes in a single step. |
| Outcome: | The proposed model improves on three datasets and can predict multi-step denoising outcomes in a single step. |
Private Synthetic Text Generation with Diffusion Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Recent research shows diffusion models are capable of generating synthetics texts . but are they also good in generating private data if the training was under differential privacy? |
| Approach: | They examine whether diffusion models are capable of generating synthetics texts . they critically assess previous work on private text generation with diffusion models . |
| Outcome: | The proposed model outperforms auto-regressive models in generating private images despite unmet privacy assumptions . the proposed model is open-source and can be used for other purposes . |
Addressing the Training-Inference Discrepancy in Discrete Diffusion for Text Generation (2025.coling-main)
Copied to clipboard
| Challenge: | Existing discrete diffusion models for text generation have a discrepancy between training and inference. |
| Approach: | They propose a training schema that considers two-step diffusion processes and a scheduling technique that gradually increases the probability of using self-generated text as training progresses. |
| Outcome: | The proposed training schema and scheduling technique improve diffusion models on four widely used datasets. |
LanguageFlow: Advancing Diffusion Language Generation with Probabilistic Flows (2024.naacl-long)
Copied to clipboard
| Challenge: | Recent work has demonstrated success in controlling sentence attributes and structure based on diffusion language models. |
| Approach: | They propose a language-rectified flow method that reformulates standard probabilistic flow models to learn ordinary differential equations to transport between the source and target distributions. |
| Outcome: | The proposed method outperforms baselines on three fine-grained control tasks and multiple high-quality text editing tasks. |
SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Control (2023.acl-long)
Copied to clipboard
| Challenge: | Existing diffusion models for continuous-valued domains have not been adopted for text data. |
| Approach: | They propose a diffusion-based language model with two key design choices . semi-autoregressive model generates blocks of text and allows local context updates . they evaluate it on unconstrained text generation benchmarks . |
| Outcome: | The proposed model outperforms autoregressive models on unconstrained text generation benchmarks on uncontrolled text generation. |
Segment-Level Diffusion: A Framework for Controllable Long-Form Generation with Diffusion Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Diffusion models have shown promise in text generation, but often struggle with generating long, coherent, and contextually accurate text. |
| Approach: | They propose a framework that enhances diffusion-based text generation through text segmentation, robust representation training with adversarial and contrastive learning, and improved latent-space guidance. |
| Outcome: | The proposed framework improves diffusion-based text generation and improves scalability and fluency. |