latent-GLAT: Glancing at Latent Variables for Parallel Text Generation (2022.acl-long)
Copied to clipboard
| Challenge: | Recent advances in text generation have limited applications due to multimodality problem. |
| Approach: | They propose a method which uses latent variables to capture word categorical information and invoke an advanced curriculum learning technique to overcome multi-modality problem. |
| Outcome: | The proposed method outperforms strong baselines without an autoregressive model, which further broadens the application scenarios of the parallel decoding paradigm. |
Similar Papers
Diffusion Glancing Transformer for Parallel Sequence-to-Sequence Learning (2024.naacl-long)
Copied to clipboard
| Challenge: | Experimental results show that non-autoregressive generation models are superior in generation efficiency but inferior in generation quality. |
| Approach: | They propose a diffusion glancing transformer which employs a modality diffusion process and residual glancy sampling to improve multi-modality modeling. |
| Outcome: | The proposed model outperforms autoregressive and non-autoregressive models on machine translation and text generation benchmarks. |
Better Exploiting Latent Variables in Text Modeling (P19-1)
Copied to clipboard
| Challenge: | Consistent gains in performance on two datasets, Penn Treebank and Yahoo, indicate the generalizability of our method. |
| Approach: | They propose a method to exploit latent variables through hidden state averaging by sampling latent variable multiple times at a gradient step. |
| Outcome: | The proposed method shows consistent gains on two datasets showing that it is generalizable. |
De-Confounded Variational Encoder-Decoder for Logical Table-to-Text Generation (2021.acl-long)
Copied to clipboard
| Challenge: | Logical table-to-text generation is challenging where deep learning models capture surface-level spurious correlations rather than the causal relationships between the table x and the sentence y. |
| Approach: | They propose to use variational inference to estimate the confounders in the latent space and cooperate with the causal intervention based on Pearl’s do-calculus to alleviate the spurious correlations. |
| Outcome: | The proposed model outperforms baselines and achieves new state-of-the-art performance on two logical table-to-text datasets in terms of logical fidelity. |
Latent-Variable Generative Models for Data-Efficient Text Classification (D19-1)
Copied to clipboard
| Challenge: | Generative classifiers offer potential advantages over discriminative classifications, including data efficiency and zero-shot learning. |
| Approach: | They introduce discrete latent variables into generative story to improve classifiers' performance . they empirically characterize performance of their models on six text classification datasets . |
| Outcome: | The proposed model outperforms discriminative and generative classifiers on six text classification datasets. |
Learning Latent Semantic Annotations for Grounding Natural Language to Structured Data (D18-1)
Copied to clipboard
| Challenge: | Existing work on grounded language learning does not capture the semantics of correspondences between structured world state representations and texts. |
| Approach: | They propose to learn explicit latent semantic annotations from paired structured tables and texts . they use an adapted semi-hidden Markov model to impose a soft constraint to further improve performance . |
| Outcome: | The proposed framework improves on a semi-hidden Markov model and extracts templates for language generation. |
CreditDecoding: Accelerating Parallel Decoding in Diffusion Large Language Models with Trace Credit (2026.acl-long)
Copied to clipboard
| Challenge: | Diffusion large language models generate text through iterative denoising with bidirectional attention, enabling richer contextual dependencies. |
| Approach: | They propose a training-free parallel decoding method that fuses Trace Credit with current logits to boost the confidence of correct but underconfident tokens. |
| Outcome: | The proposed method achieves 5.48 times speedup with +0.48 accuracy on LLaDA-8B and is orthogonal to mainstream inference optimizations. |
Generative Text Modeling through Short Run Inference (2021.eacl-main)
Copied to clipboard
| Challenge: | Latent variable models for text capture global semantic and syntactic features when trained correctly. |
| Approach: | They propose a short run dynamics for inference that initializes from the prior distribution of the latent variable and runs a small number of Langevin dynamics steps guided by its posterior distribution. |
| Outcome: | The proposed model is able to generate coherent sentences with smooth transition and shows no sign of posterior collapse. |
Twist Decoding: Diverse Generators Guide Each Other (2022.emnlp-main)
Copied to clipboard
Jungo Kasai, Keisuke Sakaguchi, Ronan Le Bras, Hao Peng, Ximing Lu, Dragomir Radev, Yejin Choi, Noah A. Smith
| Challenge: | Using a variety of language generation models, ensembling models is challenging during inference. |
| Approach: | They propose a method that decodes text models that do not assume a shared vocabulary, tokenization or generation order. |
| Outcome: | The proposed method outperforms models decoded in isolation over various scenarios. |
Fine-grained Contrastive Learning for Definition Generation (2022.aacl-main)
Copied to clipboard
| Challenge: | Recent pre-trained transformer-based definition generation models lack effective representation learning to contain full semantic components of the given word, leading to under-specific definitions. |
| Approach: | They propose a novel contrastive learning method that encourages the model to capture more detailed semantic representations from the definition sequence encoding. |
| Outcome: | The proposed method could generate more specific definitions compared with state-of-the-art models. |
Long Text Generation with Topic-aware Discrete Latent Variable Model (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent work focuses on the modeling of discourse relation, resulting in discrete codes learning shallow semantics. |
| Approach: | They propose a topic-aware latent code-guided text generation model that encourages discrete codes to model information about topics. |
| Outcome: | The proposed model generates more topic-relevant and coherent texts. |