Pretrained Language Models for Dialogue Generation with Multiple Input Sources (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Large-scale pretrained language models have achieved outstanding performance on natural language understanding tasks. |
| Approach: | They propose to fuse attention information from multiple input sources to achieve better relevance with dialogue history than simple fusion baselines. |
| Outcome: | The proposed models deliver higher relevance with dialogue history than baselines. |
Similar Papers
Knowledge-Grounded Dialogue Generation with Pre-trained Language Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Empirical results indicate that pre-trained language models can significantly outperform state-of-the-art methods in both automatic evaluation and human judgment. |
| Approach: | They propose to equip a pre-trained language model with a knowledge selection module to generate knowledge-grounded dialogues. |
| Outcome: | The proposed model outperforms state-of-the-art methods in evaluation and human judgment. |
Hello, It’s GPT-2 - How Can I Help You? Towards the Use of Pretrained Language Models for Task-Oriented Dialogue Systems (D19-56)
Copied to clipboard
| Challenge: | Statistical conversational systems are complex, timeintensive, expensive, and not easily transferable due to data scarcity. |
| Approach: | They propose a task-oriented dialogue model that operates on text input . they validate it on multi-domain task-orientated dialogues from a multi-word dataset . |
| Outcome: | The proposed model bypasses explicit policy and language generation modules on multi-domain task-oriented dialogues from the MultiWOZ dataset. |
PLATO: Pre-trained Dialogue Generation Model with Discrete Latent Variable (2020.acl-main)
Copied to clipboard
| Challenge: | Existing pre-training models for dialogue generation have been proven effective for a wide range of tasks. |
| Approach: | They propose a dialogue generation pre-training framework that leverages bi-directional context and uni-directional characteristic of language generation. |
| Outcome: | The proposed framework is superior to existing models on three publicly available datasets. |
Video-Grounded Dialogues with Pretrained Generation Language Models (2020.acl-main)
Copied to clipboard
| Challenge: | Pre-trained language models have shown success in improving downstream NLP tasks . pre-tuned models capture textual dependencies in text data of rich semantics . |
| Approach: | They propose a framework for improving video-grounded dialogue by extending GPT-2 models . they propose to combine visual and textual representation into a structured sequence . |
| Outcome: | The proposed framework improves audio-visual scene-aware dialogues benchmark on AVSD . it is based on a large pre-trained GPT-2 network and can generate natural responses . |
An Investigation of Suitability of Pre-Trained Language Models for Dialogue Generation – Avoiding Discrepancies (2021.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained language models have been widely used in open-domain dialogue generation. |
| Approach: | They propose to use decoder-only architecture to achieve excellent performance for dialogue generation. |
| Outcome: | The proposed frameworks are based on transformer-ED, transformer-Dec, transformer MLM and transformer-AR. |
Towards Efficient Dialogue Pre-training with Transferable and Interpretable Latent Structure (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing models that use millions of parameters on massive data are inefficient and lack interpretability. |
| Approach: | They propose a model with a latent structure that is easily transferable from the general domain to downstream tasks in a lightweight and transparent way. |
| Outcome: | The proposed model performs better than four strong baseline models in terms of automatic and human evaluations and is 5x faster than the strongest baseline model. |
EM Pre-training for Multi-party Dialogue Response Generation (2023.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to pretrain large language models for dialogue response generation are difficult due to the lack of annotated addressee labels in multi-party dialogue datasets. |
| Approach: | They propose an Expectation-Maximization approach that iteratively performs expectation steps to generate addressee labels and maximize a response generation model. |
| Outcome: | The proposed method is based on two-party dialogues and multi-party dialogs. |
Automatic Generation of Large-scale Multi-turn Dialogues from Reddit (2022.coling-1)
Copied to clipboard
| Challenge: | Using a set of algorithms, we can generate large dialogue corpus from Reddit. |
| Approach: | They propose to automatically convert posts and their comments from discussion forums such as Reddit into multi-turn dialogues. |
| Outcome: | The proposed methods improve on the baseline method by 36.3% . the best method shows an improvement of 36.6% over the previous one . |
Automatic Dialogue Generation with Expressed Emotions (N18-2)
Copied to clipboard
| Challenge: | a growing interest in neural dialogue generation systems is focusing on generating human-like responses based on past utterances . despite efforts, few consider putting restrictions on the response itself . authors present three models that concatenate the desired emotion with the source input . |
| Approach: | They propose three models that concatenate the desired emotion with the source input or push the emotion in the decoder. |
| Outcome: | The proposed model is more efficient than the previous models, but it lacks the emotion vector. |
Multi-Source Multi-Type Knowledge Exploration and Exploitation for Dialogue Generation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing models focus on identifying specific types of dialogue knowledge and utilizing corresponding datasets for training, but lack generalization capabilities and computational resources. |
| Approach: | They propose a framework that explores multi-source multi-type knowledge from LLMs by leveraging diverse datasets and exploits it for response generation. |
| Outcome: | The proposed framework exploits multi-source multi-type knowledge from LLMs to generate coherent, informative, and fluent responses. |