PEK: A Parameter-Efficient Framework for Knowledge-Grounded Dialogue Generation (2024.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained language models have shown great dialogue generation capability in different scenarios, but the huge VRAM consumption when fine-tuning them is one of their drawbacks. |
| Approach: | They propose a parameter-efficient framework for knowledge-enhanced dialogue generation that leverages external knowledge documents and knowledge graphs to enhance its generation capabilities. |
| Outcome: | The proposed framework outperforms baseline methods on multiple evaluation metrics on Wizard of Wikipedia and CMU_DoG datasets. |
Similar Papers
Section-Aware Commonsense Knowledge-Grounded Dialogue Generation with Pre-trained Language Model (2022.coling-1)
Copied to clipboard
| Challenge: | Pre-trained language models can be expected to deepen the fusing of dialogue context and knowledge because of their superior ability of semantic understanding. |
| Approach: | They propose a two-stage framework to integrate a linearized knowledge into plan text using a ranking network PriorRanking to estimate the relevance of a retrieved knowledge fact. |
| Outcome: | The proposed framework improves the performance of pre-trained language models by using section-aware strategies to encode the linearized knowledge. |
Knowledge-Grounded Dialogue Generation with Pre-trained Language Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Empirical results indicate that pre-trained language models can significantly outperform state-of-the-art methods in both automatic evaluation and human judgment. |
| Approach: | They propose to equip a pre-trained language model with a knowledge selection module to generate knowledge-grounded dialogues. |
| Outcome: | The proposed model outperforms state-of-the-art methods in evaluation and human judgment. |
Knowledge-Grounded Dialogue Generation with a Unified Knowledge Representation (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing knowledge-grounded dialogue systems perform poorly on unseen topics due to limited topics covered in training data. |
| Approach: | They propose a language model that homogenizes different knowledge sources to a unified knowledge representation for knowledge-grounded dialogue generation tasks. |
| Outcome: | The proposed language model generalizes well across knowledge-grounded dialogue tasks. |
A Synthetic Data Generation Framework for Grounded Dialogues (2023.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to train grounded dialogues require large amounts of data. |
| Approach: | They propose a synthetic data generation framework for grounded dialogues that takes knowledge data and heuristics to determine a dialogue flow and incrementally turn it into a dialog. |
| Outcome: | The proposed framework significantly boosts model performance in training data and low-resource scenarios. |
Efficient Latent Variable Modeling for Knowledge-Grounded Dialogue Generation (2023.findings-emnlp)
Copied to clipboard
Gunsoo Han, Daejin Jo, Daniel Nam, Eunseop Yoon, Taehwan Kwon, Seungeun Rho, Kyoung-Woon On, Chang Yoo, Sungwoong Kim
| Challenge: | Existing knowledge-grounded dialogue generation algorithms require annotated knowledge to generate a response grounded on the retrieved knowledge. |
| Approach: | They propose an efficient algorithm for latent variable modeling that leverages large amount of dialogue data. |
| Outcome: | The proposed algorithm outperforms the supervised learning algorithm on knowledge-grounded dialogue datasets while maintaining efficiency and scalability. |
Diverse and Informative Dialogue Generation with Context-Specific Commonsense Knowledge Awareness (2020.acl-main)
Copied to clipboard
| Challenge: | Generative dialogue systems tend to produce generic and boring responses, causing boring conversations . a novel commonsense knowledge-aware dialogue generation model is proposed to solve this problem . |
| Approach: | They propose to retrieve and introduce knowledge facts from knowledge graphs to reduce boring conversations . they use a Felicitous Fact mechanism to help the model focus on context-relevant knowledge facts . |
| Outcome: | The proposed model outperforms the state-of-the-art approach in most experiments. |
Learning Knowledge Bases with Parameters for Task-Oriented Dialogue Systems (2020.findings-emnlp)
Copied to clipboard
Andrea Madotto, Samuel Cahyawijaya, Genta Indra Winata, Yan Xu, Zihan Liu, Zhaojiang Lin, Pascale Fung
| Challenge: | End-to-end systems rely on dialogue state tracking and annotations to fulfill user requests . modularized systems require multiple steps, including a direct interaction with the KB . |
| Approach: | They propose a method to embed the KB directly into the model parameters . they evaluate five task-oriented dialogue datasets with small, medium, and large KBs . |
| Outcome: | The proposed model can embed the KB directly into the model parameters without any DST or template responses, nor the kb as input. |
Towards Fewer Hallucinations in Knowledge-Grounded Dialogue Generation via Augmentative and Contrastive Knowledge-Dialogue (2023.acl-short)
Copied to clipboard
| Challenge: | Existing knowledge-grounded dialogue generation models face the hallucination problem . Existing models generate inappropriate knowledge and generate inconsistent responses . |
| Approach: | They propose an Augmentative and Contrastive Knowledge Dialogue Expansion Framework to enhance existing knowledge dialogue models by polarizing optimization objectives and weak knowledge generation ability. |
| Outcome: | The proposed framework expands existing training sets and smooths the optimization objective that enables models to generate ground-truth with or without gold knowledge. |
PLATO: Pre-trained Dialogue Generation Model with Discrete Latent Variable (2020.acl-main)
Copied to clipboard
| Challenge: | Existing pre-training models for dialogue generation have been proven effective for a wide range of tasks. |
| Approach: | They propose a dialogue generation pre-training framework that leverages bi-directional context and uni-directional characteristic of language generation. |
| Outcome: | The proposed framework is superior to existing models on three publicly available datasets. |
Multi-Stage Prompting for Knowledgeable Dialogue Generation (2022.findings-acl)
Copied to clipboard
Zihan Liu, Mostofa Patwary, Ryan Prenger, Shrimai Prabhumoye, Wei Ping, Mohammad Shoeybi, Bryan Catanzaro
| Challenge: | Existing knowledge-grounded dialogue systems typically use finetuned versions of a pretrained language model and large-scale knowledge bases. |
| Approach: | They propose a multi-stage prompting approach to generate knowledgeable responses from a single pretrained LM. |
| Outcome: | The proposed model outperforms the state-of-the-art retrieval-based model in terms of knowledge relevance and correctness by 5.8% and 5%, respectively. |