Towards Efficient Dialogue Pre-training with Transferable and Interpretable Latent Structure (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing models that use millions of parameters on massive data are inefficient and lack interpretability. |
| Approach: | They propose a model with a latent structure that is easily transferable from the general domain to downstream tasks in a lightweight and transparent way. |
| Outcome: | The proposed model performs better than four strong baseline models in terms of automatic and human evaluations and is 5x faster than the strongest baseline model. |
Similar Papers
PLATO: Pre-trained Dialogue Generation Model with Discrete Latent Variable (2020.acl-main)
Copied to clipboard
| Challenge: | Existing pre-training models for dialogue generation have been proven effective for a wide range of tasks. |
| Approach: | They propose a dialogue generation pre-training framework that leverages bi-directional context and uni-directional characteristic of language generation. |
| Outcome: | The proposed framework is superior to existing models on three publicly available datasets. |
Efficient Latent Variable Modeling for Knowledge-Grounded Dialogue Generation (2023.findings-emnlp)
Copied to clipboard
Gunsoo Han, Daejin Jo, Daniel Nam, Eunseop Yoon, Taehwan Kwon, Seungeun Rho, Kyoung-Woon On, Chang Yoo, Sungwoong Kim
| Challenge: | Existing knowledge-grounded dialogue generation algorithms require annotated knowledge to generate a response grounded on the retrieved knowledge. |
| Approach: | They propose an efficient algorithm for latent variable modeling that leverages large amount of dialogue data. |
| Outcome: | The proposed algorithm outperforms the supervised learning algorithm on knowledge-grounded dialogue datasets while maintaining efficiency and scalability. |
Hello, It’s GPT-2 - How Can I Help You? Towards the Use of Pretrained Language Models for Task-Oriented Dialogue Systems (D19-56)
Copied to clipboard
| Challenge: | Statistical conversational systems are complex, timeintensive, expensive, and not easily transferable due to data scarcity. |
| Approach: | They propose a task-oriented dialogue model that operates on text input . they validate it on multi-domain task-orientated dialogues from a multi-word dataset . |
| Outcome: | The proposed model bypasses explicit policy and language generation modules on multi-domain task-oriented dialogues from the MultiWOZ dataset. |
Knowledge-Grounded Dialogue Generation with Pre-trained Language Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Empirical results indicate that pre-trained language models can significantly outperform state-of-the-art methods in both automatic evaluation and human judgment. |
| Approach: | They propose to equip a pre-trained language model with a knowledge selection module to generate knowledge-grounded dialogues. |
| Outcome: | The proposed model outperforms state-of-the-art methods in evaluation and human judgment. |
Pretrained Language Models for Dialogue Generation with Multiple Input Sources (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Large-scale pretrained language models have achieved outstanding performance on natural language understanding tasks. |
| Approach: | They propose to fuse attention information from multiple input sources to achieve better relevance with dialogue history than simple fusion baselines. |
| Outcome: | The proposed models deliver higher relevance with dialogue history than baselines. |
Pre-training Multi-party Dialogue Models with Latent Discourse Inference (2023.acl-long)
Copied to clipboard
| Challenge: | Existing studies have failed to scale up the pre-training process by putting aside unlabeled data . et al., 2019: multi-party dialogues are more difficult for models to understand since they involve multiple interlocutors resulting in interweaving reply-to relations and information flows. |
| Approach: | They propose to treat discourse structures as latent variables and jointly infer them to pre-train a model that understands the discourse structure of multi-party dialogues. |
| Outcome: | The proposed model outperforms baselines and achieves state-of-the-art results on multiple downstream tasks. |
DIALOGPT : Large-Scale Generative Pre-training for Conversational Response Generation (2020.acl-demos)
Copied to clipboard
Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, Bill Dolan
| Challenge: | DIALOGPT is a large, tunable neural conversational response generation model . trained on 147M conversation-like exchanges extracted from Reddit comment chains . |
| Approach: | They present a large, tunable neural conversational response generation model, DIALOGPT . the model is trained on 147M conversation-like exchanges extracted from Reddit comment chains . |
| Outcome: | The proposed model can generate more relevant, contentful and context-consistent responses than baseline systems. |
Sequence-to-Sequence Learning for Task-oriented Dialogue with Dialogue State Representation (C18-1)
Copied to clipboard
| Challenge: | Existing pipeline models for task-oriented dialogue system require explicit modeling of dialogue states and hand-crafted action spaces to query domain-specific knowledge base. |
| Approach: | They propose a framework that leverages the advantages of classic pipeline and sequence-to-sequence models. |
| Outcome: | The proposed framework outperforms baseline models on automatic and human evaluation on a Stanford Multi-turn Multi-domain task-oriented dialogue dataset. |
Low-Resource Dialogue Summarization with Domain-Agnostic Multi-Source Pretraining (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for low-resource dialogue summarization neglect the difference between dialogues and conventional articles. |
| Approach: | They propose a multi-source pretraining paradigm to leverage external summary data . they exploit large-scale in-domain non-summary data to separate dialogue encoder and summary decoder . |
| Outcome: | The proposed model can be used to better leverage external summary data. |
Conversations Are Not Flat: Modeling the Dynamic Information Flow across Dialogue Utterances (2021.acl-long)
Copied to clipboard
| Challenge: | Recent intelligent open-domain chatbots have made substantial progress thanks to the rapid development of large-scale pre-training approaches. |
| Approach: | They propose a dynamic flow mechanism to model the context flow and a model to capture the information dynamics across dialogue utterances. |
| Outcome: | The proposed model outperforms the DialoGPT on the dialogue generation task. |