Diversity-aware Event Prediction based on a Conditional Variational Autoencoder with Reconstruction

Challenge:	Typical event sequences are important class of commonsense knowledge . previous work in event prediction uses sequence-to-sequence models . however, what can happen after a given event is usually diverse .
Approach:	They propose to incorporate a conditional variational autoencoder into seq2seq for its ability to represent diverse next events as a probabilistic distribution.
Outcome:	The proposed model outperforms deterministic models in terms of precision and recall . the proposed model is based on a conditional variational autoencoder .

Similar Papers

Overview

Challenge

Approach

Outcome

Modeling Event Background for If-Then Commonsense Reasoning Using Context-aware Variational Autoencoder (D19-1)

Copied to clipboard

Li Du, Xiao Ding, Ting Liu, Zhongyang Li

Reasoning

Challenge:	Understanding event and event-centered commonsense reasoning is crucial for natural language processing (NLP).
Approach:	They propose a If-Then commonsense reasoning dataset Atomic and an RNN-based Seq2Seq model to facilitate this.
Outcome:	The proposed model improves the accuracy and diversity of inferences compared with baseline methods.

Variational Autoregressive Decoder for Neural Response Generation (D18-1)

Copied to clipboard

Jiachen Du, Wenjie Li, Yulan He, Ruifeng Xu, Lidong Bing, Xuan Wang

Text Generation

Challenge:	Existing variational Bayesian models generate responses from a single latent variable, which is not sufficient to model high variability in responses.
Approach:	They propose a conditional variable auto-encoder that sequentially introduces latent variables to condition the generation of each word in the response sequence.
Outcome:	Empirical results show that the proposed model improves on state-of-the-art models on Opensubtitle and Reddit datasets.

Diversifying Reply Suggestions Using a Matching-Conditional Variational Autoencoder (N19-2)

Copied to clipboard

Budhaditya Deb, Peter Bailey, Milad Shokouhi

Language Models

Challenge:	Automated reply suggestions (SR) are becoming common in many popular applications such as Gmail (2016) .
Approach:	They propose a constrained-sampling approach to make the variational inference efficient for a commercial instant-messaging system.
Outcome:	The proposed model increases diversity without losing relevance in offline experiments.

Incorporating Causal Analysis into Diversified and Logical Response Generation (2022.coling-1)

Copied to clipboard

Jiayi Liu, Wei Wei, Zhixuan Chu, Xing Gao, Ji Zhang, Tan Yan, Yulin Kang

Text Generation

Challenge:	Existing generation-based models generate generic and safe responses such as "So am I" or "I don't know"
Approach:	They propose to predict the mediators to preserve relevant information and auto-regressively incorporate the mediator into generating process.
Outcome:	The proposed model generates relevant and informative responses and outperforms the state-of-the-art in terms of automatic metrics and human evaluations.

Recurrence Boosts Diversity! Revisiting Recurrent Latent Variable in Transformer-Based Variational AutoEncoder for Diverse Text Generation (2022.findings-emnlp)

Copied to clipboard

Jinyi Hu, Xiaoyuan Yi, Wenhao Li, Maosong Sun, Xing Xie

Language Models

Challenge:	Variational Auto-Encoder (VAE) has been widely adopted in text generation due to its ability to learn flexible representations.
Approach:	They propose a Transformer-based recurrent VAE structure that imposes recurrence on segment-wise latent variables with arbitrarily separated text segments and constructs the posterior distribution with residual parameterization.
Outcome:	The proposed structure can deduce a non-zero lower bound of the KL term and enhance the entanglement of each segment and preceding latent variables, providing a theoretical guarantee of generation diversity.

Enhancing NER by Harnessing Multiple Datasets with Conditional Variational Autoencoders (2025.acl-short)

Copied to clipboard

Taku Oi, Makoto Miwa

Language Models

Challenge:	Named Entity Recognition (NER) is a fundamental NLP task . supervised learning or full fine-tuning remains essential for high performance NER models.
Approach:	They propose to integrate CVAE into a span-based Named Entity Recognition model.
Outcome:	The proposed method achieves better performance on the BioRED dataset.

Variational Attention for Sequence-to-Sequence Models (C18-1)

Copied to clipboard

Hareesh Bahuleyan, Lili Mou, Olga Vechtomova, Pascal Poupart

Language Models

Challenge:	Existing variational autoencoders encode data to latent variables and then decode them into target data.
Approach:	They propose a variational attention mechanism where the attention vector is also modeled as Gaussian distributed random variables.
Outcome:	The proposed method reduces the variational latent space bypassing phenomenon as it increases diversity of generated sentences.

Generating Relevant and Coherent Dialogue Responses using Self-Separated Conditional Variational AutoEncoders (2021.acl-long)

Copied to clipboard

Bin Sun, Shaoxiong Feng, Yiwei Li, Jiamou Liu, Kan Li

Dialogue Systems & Conversational Agents

Challenge:	Conditional Variational AutoEncoders (CVAE) can enhance the diversity and informativeness of responses in open-domain dialogue generation tasks.
Approach:	They propose a Conditional Variational AutoEncoder (CVAE) that regularizes latent variables and introduces group information to regularize them.
Outcome:	Empirical results show that the proposed model can significantly boost responses in well-established open-domain dialogue datasets.

Evidence-Aware Inferential Text Generation with Vector Quantised Variational AutoEncoder (2020.acl-main)

Copied to clipboard

Daya Guo, Duyu Tang, Nan Duan, Jian Yin, Daxin Jiang, Ming Zhou

Semantic Text Processing

Challenge:	Existing approaches for inferential text generation ignore context that is not explicitly provided . Existing models ignore background knowledge that provides crucial evidence to generate inferences .
Approach:	They propose an approach that automatically finds evidence for an event from a large text corpus and leverages it to guide the generation of inferential texts.
Outcome:	The proposed model generates inferential texts from a large text corpus and uses evidence to guide it.

Conditional Generation of Temporally-ordered Event Sequences (2021.acl-long)

Copied to clipboard

Shih-Ting Lin, Nathanael Chambers, Greg Durrett

Language Models

Challenge:	a new model of narrative schema knowledge does not capture the temporal relationships between events . a temporal order model is able to unscramble event sequences without access to labeled temporal training data .
Approach:	They propose a temporal order-based model that can be flexibly applied to different tasks . they use a BART-based conditional generation model that captures temporal co-occurrence .
Outcome:	The proposed model outperforms existing models on temporal ordering and event infilling tasks.