DialogVED: A Pre-trained Latent Variable Encoder-Decoder Model for Dialog Response Generation (2022.acl-long)
Copied to clipboard
Wei Chen, Yeyun Gong, Song Wang, Bolun Yao, Weizhen Qi, Zhongyu Wei, Xiaowu Hu, Bartuer Zhou, Yi Mao, Weizhu Chen, Biao Cheng, Nan Duan
| Challenge: | Existing pre-trained dialog models shed light on various downstream tasks in natural language processing (NLP). |
| Approach: | They propose a dialog pre-training framework that introduces latent variables into the enhanced encoder-decoder pre-train framework to increase relevance and diversity of responses. |
| Outcome: | The proposed model achieves state-of-the-art on personaChat, DailyDialog, and DSTC7-AVSD datasets. |
Similar Papers
DIALOGPT : Large-Scale Generative Pre-training for Conversational Response Generation (2020.acl-demos)
Copied to clipboard
Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, Bill Dolan
| Challenge: | DIALOGPT is a large, tunable neural conversational response generation model . trained on 147M conversation-like exchanges extracted from Reddit comment chains . |
| Approach: | They present a large, tunable neural conversational response generation model, DIALOGPT . the model is trained on 147M conversation-like exchanges extracted from Reddit comment chains . |
| Outcome: | The proposed model can generate more relevant, contentful and context-consistent responses than baseline systems. |
Unsupervised Discrete Sentence Representation Learning for Interpretable Neural Dialog Generation (P18-1)
Copied to clipboard
| Challenge: | Existing encoder-decoder dialog models cannot output interpretable actions as in traditional systems. |
| Approach: | They propose an unsupervised discrete sentence representation learning method that integrates with existing encoder-decoder dialog models for interpretable response generation. |
| Outcome: | The proposed model can be integrated with existing encoder-decoder dialog models and discover interpretable semantics via either auto encoding or context predicting. |
PLATO: Pre-trained Dialogue Generation Model with Discrete Latent Variable (2020.acl-main)
Copied to clipboard
| Challenge: | Existing pre-training models for dialogue generation have been proven effective for a wide range of tasks. |
| Approach: | They propose a dialogue generation pre-training framework that leverages bi-directional context and uni-directional characteristic of language generation. |
| Outcome: | The proposed framework is superior to existing models on three publicly available datasets. |
Dior-CVAE: Pre-trained Language Models and Diffusion Priors for Variational Dialog Generation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing variational dialog models have pre-trained, restricting diversity of responses . a diffusion model increases complexity of prior distribution and its compatibility with PLMs . |
| Approach: | They propose a hierarchical conditional variational autoencoder with diffusion priors to address these challenges. |
| Outcome: | The proposed method generates more diverse responses without dialog pre-training. |
Adversarial Learning on the Latent Space for Diverse Dialog Generation (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods for dialog generation generate generic utterances, e.g., always generating "I don't know" |
| Approach: | They propose a framework that uses generative adversarial nets to generate conditioned responses in dialogs. |
| Outcome: | The proposed model generates more fluent, relevant, and diverse responses than state-of-the-art methods. |
DiffusionDialog: A Diffusion Model for Diverse Dialog Generation with Latent Space (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing studies have tried to introduce discrete or Gaussian-based latent variables to address the one-to-many problem, but the diversity is limited. |
| Approach: | They propose a diffusion model to enhance the diversity of dialogue generation by using continuous latent variables instead of discrete ones. |
| Outcome: | The proposed model greatly enhances diversity of dialog response while keeping the coherence. |
Language Model as an Annotator: Exploring DialoGPT for Dialogue Summarization (2021.acl-long)
Copied to clipboard
| Challenge: | Existing dialogue summarization systems encode text with a number of general semantic features, but these are often not available in open-domain tools. |
| Approach: | They propose to use DialoGPT to label three types of features on two datasets . they propose to employ pre-trained and non-pre-tried models as dialogue annotators . |
| Outcome: | The proposed method improves on two dialogue summarization datasets and achieves state-of-the-art performance. |
Boosting Dialog Response Generation (P19-1)
Copied to clipboard
| Challenge: | Neural models generate the most common and generic responses all the time . Empirical results show that our method can significantly improve the diversity of responses generated by sequence-to-sequence models. |
| Approach: | They propose an iterative training process and ensemble method based on boosting to improve the diversity of responses generated by neural models. |
| Outcome: | Empirical results show that the proposed method significantly improves diversity and relevance of responses generated by all models. |
EM Pre-training for Multi-party Dialogue Response Generation (2023.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to pretrain large language models for dialogue response generation are difficult due to the lack of annotated addressee labels in multi-party dialogue datasets. |
| Approach: | They propose an Expectation-Maximization approach that iteratively performs expectation steps to generate addressee labels and maximize a response generation model. |
| Outcome: | The proposed method is based on two-party dialogues and multi-party dialogs. |
An Investigation of Suitability of Pre-Trained Language Models for Dialogue Generation – Avoiding Discrepancies (2021.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained language models have been widely used in open-domain dialogue generation. |
| Approach: | They propose to use decoder-only architecture to achieve excellent performance for dialogue generation. |
| Outcome: | The proposed frameworks are based on transformer-ED, transformer-Dec, transformer MLM and transformer-AR. |