Papers with auto-regressive
ELMER: A Non-Autoregressive Pre-trained Language Model for Efficient and Effective Text Generation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for text generation use auto-regressive (AR) methods, but inefficient inference is a problem. |
| Approach: | They propose an efficient and effective PLM to explicitly model the token dependency during NAR text generation. |
| Outcome: | The proposed model outperforms existing models on three text generation tasks while achieving 10 times faster inference speedup. |
Facts2Story: Controlling Text Generation by Key Facts (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods for story generation struggle with staying coherent for long periods of time. |
| Approach: | They propose a controlled generation task which expands a sequence of facts into a longer narrative. |
| Outcome: | The proposed model produces competitive fluency while adhering to the requested facts. |
GECSum: Generative Evaluation-Driven Sequence Level Contrastive Learning for Abstractive Summarization (2024.lrec-main)
Copied to clipboard
| Challenge: | Abstractive summarization is a technique in natural language processing that involves generating a summary of a source document by creating new sentences and phrases. |
| Approach: | They propose a sequence-level contrastive learning framework that leverages the semantic understanding capabilities of the abstractive model itself to evaluate summary in reference-based settings. |
| Outcome: | The proposed framework outperforms the state-of-the-art in four summarization datasets. |
Distilling Knowledge Learned in BERT for Text Generation (2020.acl-main)
Copied to clipboard
| Challenge: | Large-scale pre-trained language models such as BERT have revolutionized the state of the art in many language understanding tasks. |
| Approach: | They propose a conditional masked language modeling approach to fine tune BERT on target generation tasks by imposing global sequence-level supervision on conventional Seq2Seq models. |
| Outcome: | The proposed model outperforms strong Transformer baselines on multiple language generation tasks such as machine translation and text summarization. |
Lost in Diffusion: Uncovering Hallucination Patterns and Failure Modes in Diffusion Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | dLLMs have emerged as a promising non-autoregressive paradigm for text generation, but their hallucination mechanisms remain underexplored. |
| Approach: | They present the first controlled comparative study to evaluate hallucination patterns in Diffusion Large Language Models. |
| Outcome: | The proposed model exhibits higher propensity for hallucination than AR counterparts controlled for architecture, scale, and pre-training weights. |
Diffusion Guided Language Modeling (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing guidance methods for text generation are prone to decoding errors and degrade performance. |
| Approach: | They propose a model that steers an auto-regressive language model to generate text with desired properties. |
| Outcome: | The proposed model outperforms existing guidance methods on a wide range of benchmark data sets. |