Speculative Sampling in Variational Autoencoders for Dialogue Response Generation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies have tried to improve variational models but they fail to learn proper mappings. |
| Approach: | They propose to use a variable-based sampling technique to find the most probable one from redundantly sampled latent variables to tie up the variable with a given response. |
| Outcome: | The proposed method is effective in response generation with massive dialogue data constructed from Twitter posts. |
Similar Papers
Better Exploiting Latent Variables in Text Modeling (P19-1)
Copied to clipboard
| Challenge: | Consistent gains in performance on two datasets, Penn Treebank and Yahoo, indicate the generalizability of our method. |
| Approach: | They propose a method to exploit latent variables through hidden state averaging by sampling latent variable multiple times at a gradient step. |
| Outcome: | The proposed method shows consistent gains on two datasets showing that it is generalizable. |
Diversifying Reply Suggestions Using a Matching-Conditional Variational Autoencoder (N19-2)
Copied to clipboard
| Challenge: | Automated reply suggestions (SR) are becoming common in many popular applications such as Gmail (2016) . |
| Approach: | They propose a constrained-sampling approach to make the variational inference efficient for a commercial instant-messaging system. |
| Outcome: | The proposed model increases diversity without losing relevance in offline experiments. |
Variational Autoregressive Decoder for Neural Response Generation (D18-1)
Copied to clipboard
| Challenge: | Existing variational Bayesian models generate responses from a single latent variable, which is not sufficient to model high variability in responses. |
| Approach: | They propose a conditional variable auto-encoder that sequentially introduces latent variables to condition the generation of each word in the response sequence. |
| Outcome: | Empirical results show that the proposed model improves on state-of-the-art models on Opensubtitle and Reddit datasets. |
Empirical Prior for Text Autoencoders (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Variational Autoencoders (VAE) are used to train generative models with latent variables. |
| Approach: | They propose a transition from Variational Autoencoders (VAE) to text autoencodeurs (AE) which model a compact latent space and preserves the capability of the language model itself. |
| Outcome: | The proposed method generates higher quality and more diverse text than the VAE-based Transformer baselines, and is more efficient than previous approaches. |
Generating Relevant and Coherent Dialogue Responses using Self-Separated Conditional Variational AutoEncoders (2021.acl-long)
Copied to clipboard
| Challenge: | Conditional Variational AutoEncoders (CVAE) can enhance the diversity and informativeness of responses in open-domain dialogue generation tasks. |
| Approach: | They propose a Conditional Variational AutoEncoder (CVAE) that regularizes latent variables and introduces group information to regularize them. |
| Outcome: | Empirical results show that the proposed model can significantly boost responses in well-established open-domain dialogue datasets. |
A Discrete CVAE for Response Generation on Short-Text Conversation (D19-1)
Copied to clipboard
| Challenge: | Neural conversation models are easy to generate bland and generic responses . however, their improvement of generating high-quality responses is still unsatisfactory . |
| Approach: | They propose to use a discrete latent variable with an explicit semantic meaning to improve the conditional variational autoencoder on short-text conversation. |
| Outcome: | The proposed model outperforms various kinds of generation models under automatic and human evaluations and generates more diverse and informative responses. |
Explicit Bayesian Inference to Uncover the Latent Themes of Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have impressive generative capabilities, yet their inner mechanisms remain largely opaque. |
| Approach: | They propose a variational autoencoder-based neural topic model to interpret LLMs generation process through an explicit Bayesian framework by inferring latent topic variables via variational inference. |
| Outcome: | The proposed model outperforms state-of-the-art topic models on intrinsic measures of coherence and diversity on multiple datasets and shows significant gains on classification and summarization tasks. |
Guiding Variational Response Generator to Exploit Persona (2020.acl-main)
Copied to clipboard
Bowen Wu, MengYuan Li, Zongsheng Wang, Yifu Chen, Derek F. Wong, Qihang Feng, Junhong Huang, Baoxun Wang
| Challenge: | Neural Response Generators (NRGs) use persona information of users to perform personalized conversations . current studies focus on incorporating explicit meta-data of user profiles or character descriptions to generate persona-aware responses. |
| Approach: | They propose to use persona information of users in Neural Response Generators to perform personalized conversations. |
| Outcome: | The proposed method improves persona-aware response generation and the metrics are reasonable to evaluate them. |
Speculative Decoding for Multi-Sample Inference (2025.findings-emnlp)
Copied to clipboard
Yiwei Li, Jiayi Shi, Shaoxiong Feng, Peiwen Yuan, Xinglin Wang, Yueqi Zhang, Ji Zhang, Chuyi Tan, Boyuan Pan, Yao Hu, Kan Li
| Challenge: | Speculative decoding method exploits consensus of parallel reasoning paths to synthesize high-quality draft tokens without auxiliary models or external databases. |
| Approach: | They propose a speculative decoding method that exploits the consensus of parallel reasoning paths to synthesize high-quality draft tokens without auxiliary models or external databases. |
| Outcome: | The proposed method exploits the intrinsic consensus of parallel reasoning paths to synthesize high-quality draft tokens without auxiliary models or databases. |
A Hierarchical Latent Structure for Variational Conversation Modeling (N18-1)
Copied to clipboard
| Challenge: | Variational autoencoders suffer from the notorious degeneration problem, according to a new study . utterance drop regularization is an important feature of the hierarchical RNNs . |
| Approach: | They propose a variational hierarchical conversation RNN framework that exploits latent variables and an utterance drop regularization to exploit latent variable. |
| Outcome: | The proposed model outperforms state-of-the-art models on Cornell Movie Dialog and Ubuntu Dialog Corpus. |