| Challenge: | Existing methods for task-oriented dialogs require fluent natural language responses and correct external actions . but they perform poorly in real world dialog tasks, a new study shows . |
| Approach: | They propose a hybrid model where nearest neighbor is used to generate fluent responses and Seq2Seq type models ensure dialogue coherency and generate accurate external actions. |
| Outcome: | The proposed model achieves a 78% relative improvement in fluency and 200% improvement in accuracy of external calls. |
Similar Papers
Making Task-Oriented Dialogue Datasets More Natural by Synthetically Generating Indirect User Requests (2025.coling-main)
Copied to clipboard
| Challenge: | Existing task-oriented dialogue benchmarks lack sufficient examples of complex discourse phenomena such as indirectness. |
| Approach: | They propose a set of linguistic criteria and an LLM-based pipeline for generating realistic IURs to test natural language understanding and dialogue state tracking models before deployment in a new domain. |
| Outcome: | The proposed model can handle indirect user requests (IURs) but lacks examples of complex discourse phenomena such as indirectness. |
Hello, It’s GPT-2 - How Can I Help You? Towards the Use of Pretrained Language Models for Task-Oriented Dialogue Systems (D19-56)
Copied to clipboard
| Challenge: | Statistical conversational systems are complex, timeintensive, expensive, and not easily transferable due to data scarcity. |
| Approach: | They propose a task-oriented dialogue model that operates on text input . they validate it on multi-domain task-orientated dialogues from a multi-word dataset . |
| Outcome: | The proposed model bypasses explicit policy and language generation modules on multi-domain task-oriented dialogues from the MultiWOZ dataset. |
Taskmaster-1: Toward a Realistic and Diverse Dialog Dataset (D19-1)
Copied to clipboard
Bill Byrne, Karthik Krishnamoorthi, Chinnadhurai Sankar, Arvind Neelakantan, Ben Goodrich, Daniel Duckworth, Semih Yavuz, Amit Dubey, Kyu-Young Kim, Andy Cedilnik
| Challenge: | a lack of high quality conversational data is limiting progress in dialog systems . we present a dataset of 13,215 task-based dialogs . |
| Approach: | They propose a task-based dialog dataset which includes 13,215 task-related dialogs . they use a two-person, spoken "Wizard of Oz" approach and a "self-dialog" approach . |
| Outcome: | The taskmaster-1 dataset contains 13,215 task-based dialogs comprising six domains. |
CHAI: A CHatbot AI for Task-Oriented Dialogue with Offline Reinforcement Learning (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to training dialogue agents are supervised learning, but this is prohibitively expensive and time-consuming. |
| Approach: | They propose offline reinforcement learning methods that can be used to train dialogue agents . offline reinforcement learn methods can be combined with language models to yield realistic dialogue agents. |
| Outcome: | The proposed method can be combined with language models to produce realistic dialogue agents . the results show that the offline method can achieve the goal of the proposed system . |
Plug-and-Play Conversational Models (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Large conversational models that generate coherent and fluent responses often require large dialogue datasets. |
| Approach: | They propose and evaluate plug-and-play methods for controllable response generation . they demonstrate a high degree of control over the generated conversational responses . |
| Outcome: | The proposed method does not require further computation at decoding time and does not need fine-tuning of a large language model. |
Leveraging Implicit Feedback from Deployment Data in Dialogue (2024.eacl-short)
Copied to clipboard
| Challenge: | Xu et al., 2023) and Bai ed., 2019) use crowdworkers to collect signals from natural dialogue episodes. |
| Approach: | They use the publicly released BlenderBot deployment data to extract signals from conversations to implicitly measure the quality of a machine-generated utterance. |
| Outcome: | The proposed model improves over baseline models, but some proxy signals can lead to undesirable generations. |
IP-Dialog: Evaluating Implicit Personalization in Dialogue Systems with Synthetic Data (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Traditional data generation methods are labor-intensive, resource-demanding, and raise privacy concerns. |
| Approach: | They propose an automatic synthetic data generation approach and introduce the **I**mplicit **P**ersonalized **Dialog**ue benchmark along with a training dataset, covering 10 tasks and 12 user attribute types. |
| Outcome: | The proposed approach incorporates the **Implicit **P**ersonalized **Dialog**ue benchmark along with a training dataset, covering 10 tasks and 12 user attribute types. |
NatCS: Eliciting Natural Customer Support Dialogues (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing task-oriented dialogue datasets do not reflect the expected characteristics of real customer support conversations. |
| Approach: | They propose to collect real customer service conversations from real conversations . they show that dialogue act annotations provide more effective training data . |
| Outcome: | The proposed approach is more representative of real human-to-human conversations compared to existing dialogue datasets . the proposed approach can be used to facilitate open research in natural dialog systems . |
SynthDST: Synthetic Data is All You Need for Few-Shot Dialog State Tracking (2024.eacl-long)
Copied to clipboard
| Challenge: | In-context learning with Large Language Models (LLMs) is a promising avenue of research in Dialog State Tracking (DST). |
| Approach: | They propose a data generation framework tailored for Dialog State Tracking that uses large language models to synthesize natural, coherent, and free-flowing dialogues with DST annotations. |
| Outcome: | The proposed framework improves joint goal accuracy by 4-5% over the zero-shot baseline on MultiWOZ 2.1 and 2.4. |
Simulated Chats for Building Dialog Systems: Learning to Generate Conversations from Instructions (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Popular dialog datasets such as MultiWOZ are created by providing crowd workers with instructions that describe the task to be accomplished. |
| Approach: | They propose a data creation strategy that uses a pre-trained language model to simulate the interaction between crowd workers by creating a user bot and an agent bot. |
| Outcome: | The proposed data creation strategy improves on two publicly available datasets using a pre-trained language model and a smaller percentage of actual crowd-generated conversations and their corresponding instructions. |