Multi-Domain Goal-Oriented Dialogues (MultiDoGO): Strategies toward Curating and Annotating Large Scale Dialogue Data (D19-1)
Copied to clipboard
| Challenge: | a large number of goal-oriented dialogue datasets are limited in their size, linguistic diversity, domain coverage, or annotation granularity. |
| Approach: | They propose a multi-domain goal-oriented dialogue dataset that uses a crowd-sourced worker and a trained annotator to curate and annotate large scale data. |
| Outcome: | The proposed dataset is 8 times the size of the largest comparable dialogue dataset available to the public. |
Similar Papers
MultiWOZ - A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling (D18-1)
Copied to clipboard
Paweł Budzianowski, Tsung-Hsien Wen, Bo-Hsiang Tseng, Iñigo Casanueva, Stefan Ultes, Osman Ramadan, Milica Gašić
| Challenge: | a dataset of 10k human-human written conversations is one order of magnitude larger than previous annotated task-oriented corpora. |
| Approach: | They propose to collect 10k human-human written conversations from a crowd-sourced dataset using crowd-sourcing. |
| Outcome: | The proposed dataset is one order of magnitude larger than previous annotated task-oriented corpora and shows the usability of the data and sets a baseline for future studies. |
RiSAWOZ: A Large-Scale Multi-Domain Wizard-of-Oz Dataset with Rich Semantic Annotations for Task-Oriented Dialogue Modeling (2020.emnlp-main)
Copied to clipboard
| Challenge: | RiSAWOZ contains 11.2K human-to-human (H2H) multi-turn semantically annotated dialogues spanning over 12 domains . despite of substantial progress made, there are challenges in creating challenging datasets in terms of size, multiple domains, semantic annotations and complexity. |
| Approach: | They propose a large-scale multi-domain Chinese Wizard-of-Oz dataset with rich semantic annotations that captures discourse phenomena for task-oriented dialogue modeling. |
| Outcome: | The proposed dataset contains 11.2K human-to-human (H2H) multi-turn semantically annotated dialogues with more than 150K utterances spanning over 12 domains. |
MultiWOZ 2.1: A Consolidated Multi-Domain Dialogue Dataset with State Corrections and State Tracking Baselines (2020.lrec-1)
Copied to clipboard
Mihail Eric, Rahul Goel, Shachi Paul, Abhishek Sethi, Sanchit Agarwal, Shuyang Gao, Adarsh Kumar, Anuj Goyal, Peter Ku, Dilek Hakkani-Tur
| Challenge: | MultiWOZ 2.0 has substantial noise in dialogue state annotations and dialogue utterances . follow-up work has augmented the original dataset with user dialogue acts . |
| Approach: | They propose to reannotate dialogue state and utterances based on original dataset . they then compare their results to other datasets to improve their models . |
| Outcome: | The proposed dataset improves on the noise in the dialogue state annotations and dialogue utterances. |
DialogStudio: Towards Richest and Most Diverse Unified Dataset Collection for Conversational AI (2024.findings-eacl)
Copied to clipboard
Jianguo Zhang, Kun Qian, Zhiwei Liu, Shelby Heinecke, Rui Meng, Ye Liu, Zhou Yu, Huan Wang, Silvio Savarese, Caiming Xiong
| Challenge: | DialogStudio is the largest and most diverse collection of dialogue datasets . existing datasets lack diversity and comprehensiveness, authors say . |
| Approach: | They introduce DialogStudio: the largest and most diverse collection of dialogue datasets . DialogStuio aggregates more than 80 diverse dialogue dataset . |
| Outcome: | a new dataset is created to improve the quality and diversity of dialogue datasets . DialogStudio is the largest and most diverse collection of dialogue data . |
CrossWOZ: A Large-Scale Chinese Cross-Domain Task-Oriented Dialogue Dataset (2020.tacl-1)
Copied to clipboard
| Challenge: | Despite the significant contributions to the community, there is still a gap between existing dialogue corpora and real-life human dialogue data. |
| Approach: | They propose to develop Chinese cross-domain wizard-of-oz task-oriented dataset CrossWOZ with rich annotations of dialogue states and dialogue acts on both user and system sides. |
| Outcome: | The proposed dataset contains 6K dialogue sessions and 102K utterances for 5 domains, including hotel, restaurant, attraction, metro, and taxi. |
GlobalWoZ: Globalizing MultiWoZ to Develop Multilingual Task-Oriented Dialogue Systems (2022.acl-long)
Copied to clipboard
| Challenge: | Existing multilingual task-oriented dialogue datasets lack high-quality data curation due to the high expense and challenges of human annotation. |
| Approach: | They propose a method that generates a multilingual ToD dataset globalized from an English ToD data set for three unexplored use cases of multilingual toD systems. |
| Outcome: | The proposed method generates a large-scale multilingual ToD dataset globalized from an English ToD data set for three unexplored use cases of multilingual toD systems. |
A Unifying View On Task-oriented Dialogue Annotation (2022.lrec-1)
Copied to clipboard
| Challenge: | Recent research attention in task-oriented dialogue systems focuses on end-to-end neural models. |
| Approach: | They present a dataset that combines annotated corpora from four domains to provide a unified ontology and annotation schema for task-oriented dialogues. |
| Outcome: | The proposed dataset improves language, information content and performance in dialogues with two recent models. |
Target-oriented Proactive Dialogue Systems with Personalization: Problem Formulation and Dataset Curation (2023.emnlp-main)
Copied to clipboard
| Challenge: | a recent study defines a conversation target from the system side to proactively steer conversations toward predefined targets or accomplish specific system-side goals. |
| Approach: | They propose a dataset curation framework that automatically curations a large-scale personalized dialogue dataset using a role-playing approach. |
| Outcome: | The proposed dataset is of high quality and could contribute to exploring personalized target-oriented dialogue. |
JMultiWOZ: A Large-Scale Japanese Multi-Domain Task-Oriented Dialogue Dataset (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing datasets for task-oriented dialogue systems in English are limited compared to Japanese. |
| Approach: | They evaluated the dialogue state tracking and response generation capabilities of Japanese language datasets using multi-domain task-oriented dialogues. |
| Outcome: | The proposed dataset provides a benchmark that is on par with MultiWOZ2.2 and the latest large language model (LLM)-based methods. |
Multi 3 WOZ: A Multilingual, Multi-Domain, Multi-Parallel Dataset for Training and Evaluating Culturally Adapted Task-Oriented Dialog Systems (2023.tacl-1)
Copied to clipboard
Songbo Hu, Han Zhou, Mete Hergul, Milan Gritta, Guchun Zhang, Ignacio Iacobacci, Ivan Vulić, Anna Korhonen
| Challenge: | Task-oriented dialog (TOD) is one of the central objectives, hallmarks, and applications of machine intelligence. |
| Approach: | They propose a multilingual, multi-domain, multiparallele ToD dataset that offers culturally adapted dialogs in 4 languages for training and evaluation of multilingual and cross-lingual systems. |
| Outcome: | The proposed dataset is large-scale and culturally adapted to enable training and evaluation of multilingual and cross-lingual ToD systems. |