Long Text Generation with Topic-aware Discrete Latent Variable Model (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent work focuses on the modeling of discourse relation, resulting in discrete codes learning shallow semantics. |
| Approach: | They propose a topic-aware latent code-guided text generation model that encourages discrete codes to model information about topics. |
| Outcome: | The proposed model generates more topic-relevant and coherent texts. |
Similar Papers
DiscoDVT: Generating Long Text with Discourse-Aware Discrete Variational Transformer (2021.emnlp-main)
Copied to clipboard
| Challenge: | Generating long passages that maintain long-range coherence is a long-standing problem in natural language generation (NLG). |
| Approach: | They propose a discourse-aware discrete variational Transformer that learns a latent variable sequence that summarizes the global structure of the text and then applies it to guide the generation process at each decoding step. |
| Outcome: | The proposed model can generate long texts with better long-range coherence by learning a latent variable sequence with each latent code. |
Long Text Generation by Modeling Sentence-Level and Discourse-Level Coherence (2021.acl-long)
Copied to clipboard
| Challenge: | Existing generation models struggle to maintain a coherent event sequence throughout the generated text. |
| Approach: | They propose a long text generation model which can represent prefix sentences at sentence level and discourse level in the decoding process. |
| Outcome: | The proposed model can generate more coherent texts than state-of-the-art models. |
Narrative Text Generation with a Latent Discrete Plan (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Prior work on story generation has focused on generating consistent stories via story outlines using keywords or key phrases. |
| Approach: | They propose a deep latent variable model that first samples a sequence of anchor words, one per sentence in the story, as part of its generative process. |
| Outcome: | The proposed model gets favorable scores when evaluated on perplexity, diversity, and control of story via discrete plan. |
Learning Semantic Textual Similarity via Topic-informed Discrete Latent Variables (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent discrete latent variable models have received a surge of interest in both NLP and CV . they are comparable to the continuous counterparts in representation learning, but are more interpretable in their predictions. |
| Approach: | They develop a topic-informed discrete latent variable model for semantic textual similarity . they inject the quantized representation into a transformer-based language model . |
| Outcome: | The proposed model outperforms strong baselines in semantic textual similarity tasks. |
Towards Generating Long and Coherent Text with Multi-Level Latent Variable Models (P19-1)
Copied to clipboard
| Challenge: | Variational autoencoders (VAEs) have received much attention as an end-to-end architecture for text generation with latent variables. |
| Approach: | They propose to leverage several multi-level structures to learn a variational autoencoder model for generating long, and coherent text. |
| Outcome: | The proposed model produces more coherent and less repetitive long text compared to baselines and mitigates posterior collapse issue. |
Efficient Latent Variable Modeling for Knowledge-Grounded Dialogue Generation (2023.findings-emnlp)
Copied to clipboard
Gunsoo Han, Daejin Jo, Daniel Nam, Eunseop Yoon, Taehwan Kwon, Seungeun Rho, Kyoung-Woon On, Chang Yoo, Sungwoong Kim
| Challenge: | Existing knowledge-grounded dialogue generation algorithms require annotated knowledge to generate a response grounded on the retrieved knowledge. |
| Approach: | They propose an efficient algorithm for latent variable modeling that leverages large amount of dialogue data. |
| Outcome: | The proposed algorithm outperforms the supervised learning algorithm on knowledge-grounded dialogue datasets while maintaining efficiency and scalability. |
Explicit Bayesian Inference to Uncover the Latent Themes of Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have impressive generative capabilities, yet their inner mechanisms remain largely opaque. |
| Approach: | They propose a variational autoencoder-based neural topic model to interpret LLMs generation process through an explicit Bayesian framework by inferring latent topic variables via variational inference. |
| Outcome: | The proposed model outperforms state-of-the-art topic models on intrinsic measures of coherence and diversity on multiple datasets and shows significant gains on classification and summarization tasks. |
Latent-Variable Generative Models for Data-Efficient Text Classification (D19-1)
Copied to clipboard
| Challenge: | Generative classifiers offer potential advantages over discriminative classifications, including data efficiency and zero-shot learning. |
| Approach: | They introduce discrete latent variables into generative story to improve classifiers' performance . they empirically characterize performance of their models on six text classification datasets . |
| Outcome: | The proposed model outperforms discriminative and generative classifiers on six text classification datasets. |
Towards Content Transfer through Grounded Text Generation (N19-1)
Copied to clipboard
| Challenge: | Recent work in neural natural language generation has attracted significant interest in controlling the form of text, such as style, persona, and wordiness. |
| Approach: | They propose a task where the task is to generate a next sentence in a document that fits its context and is grounded in . external textual source such as a news story. |
| Outcome: | The proposed task is based on 640k Wikipedia referenced sentences paired with the source articles to show significant improvements against baselines. |
Posterior Control of Blackbox Generation (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods for conditional natural language generation are limited in their ability to produce controlled output. |
| Approach: | They propose to augment neural generation models with discrete control states learned through a structured latent-variable approach. |
| Outcome: | The proposed approach improves over benchmarks while providing fine-grained control. |