X-RiSAWOZ: High-Quality End-to-End Multilingual Dialogue Datasets and Few-shot Agents (2023.findings-acl)
Copied to clipboard
Mehrad Moradshahi, Tianhao Shen, Kalika Bali, Monojit Choudhury, Gael de Chalendar, Anmol Goel, Sungkyun Kim, Prashant Kodali, Ponnurangam Kumaraguru, Nasredine Semmar, Sina Semnani, Jiwon Seo, Vivek Seshadri, Manish Shrivastava, Michael Sun, Aditya Yadavalli, Chaobin You, Deyi Xiong, Monica Lam
| Challenge: | X-RiSAWOZ dataset has more than 18,000 human-verified dialogue utterances for each language . Xiaoping and Xinhui are the main challenges for task-oriented dialogue research . |
| Approach: | They develop a toolkit to accelerate the post-editing of a new language dataset after translation . their dataset, code, and toolkit are released open-source . |
| Outcome: | The proposed toolkit accelerates the post-editing of a new language dataset after translation. |
Similar Papers
RiSAWOZ: A Large-Scale Multi-Domain Wizard-of-Oz Dataset with Rich Semantic Annotations for Task-Oriented Dialogue Modeling (2020.emnlp-main)
Copied to clipboard
| Challenge: | RiSAWOZ contains 11.2K human-to-human (H2H) multi-turn semantically annotated dialogues spanning over 12 domains . despite of substantial progress made, there are challenges in creating challenging datasets in terms of size, multiple domains, semantic annotations and complexity. |
| Approach: | They propose a large-scale multi-domain Chinese Wizard-of-Oz dataset with rich semantic annotations that captures discourse phenomena for task-oriented dialogue modeling. |
| Outcome: | The proposed dataset contains 11.2K human-to-human (H2H) multi-turn semantically annotated dialogues with more than 150K utterances spanning over 12 domains. |
Contextual Semantic Parsing for Multilingual Task-Oriented Dialogues (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing methods for predicting state of a conversation are limited to a few languages . a method that can be applied to other languages will benefit the large population of speakers of many other languages. |
| Approach: | They propose to automatically translate large-scale dialogue data sets in one language to produce an effective semantic parser for other languages using machine translation. |
| Outcome: | The proposed model reduces the compounding effect of translation errors without harming the accuracy in practice. |
xDial-Eval: A Multilingual Open-Domain Dialogue Evaluation Benchmark (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Currently, human evaluation is the most reliable way to holistically judge the quality of the dialogue. |
| Approach: | They propose to use English dialogue evaluation metrics to generalize them to other languages. |
| Outcome: | The proposed metrics outperform OpenAI’s ChatGPT in terms of average Pearson correlations over all datasets and languages. |
GlobalWoZ: Globalizing MultiWoZ to Develop Multilingual Task-Oriented Dialogue Systems (2022.acl-long)
Copied to clipboard
| Challenge: | Existing multilingual task-oriented dialogue datasets lack high-quality data curation due to the high expense and challenges of human annotation. |
| Approach: | They propose a method that generates a multilingual ToD dataset globalized from an English ToD data set for three unexplored use cases of multilingual toD systems. |
| Outcome: | The proposed method generates a large-scale multilingual ToD dataset globalized from an English ToD data set for three unexplored use cases of multilingual toD systems. |
Multi 3 WOZ: A Multilingual, Multi-Domain, Multi-Parallel Dataset for Training and Evaluating Culturally Adapted Task-Oriented Dialog Systems (2023.tacl-1)
Copied to clipboard
Songbo Hu, Han Zhou, Mete Hergul, Milan Gritta, Guchun Zhang, Ignacio Iacobacci, Ivan Vulić, Anna Korhonen
| Challenge: | Task-oriented dialog (TOD) is one of the central objectives, hallmarks, and applications of machine intelligence. |
| Approach: | They propose a multilingual, multi-domain, multiparallele ToD dataset that offers culturally adapted dialogs in 4 languages for training and evaluation of multilingual and cross-lingual systems. |
| Outcome: | The proposed dataset is large-scale and culturally adapted to enable training and evaluation of multilingual and cross-lingual ToD systems. |
Towards more equitable question answering systems: How much more data do you need? (2021.acl-short)
Copied to clipboard
| Challenge: | Question answering datasets in English are relatively new, but lack of linguistic diversity in the field is a challenge. |
| Approach: | They propose to use translation and cross-lingual transfer to produce QA systems in multiple languages to improve their performance. |
| Outcome: | The proposed approaches take advantage of existing resources to produce QA systems in multiple languages. |
Multi2WOZ: A Robust Multilingual Dataset and Conversational Pretraining for Task-Oriented Dialog (2022.naacl-main)
Copied to clipboard
| Challenge: | Task-oriented dialog (TOD) is arguably one of the most popular natural language processing (NLP) application areas. |
| Approach: | They propose a multilingual multi-domain TOD dataset that spans four languages . they use a framework for multilingual conversational specialization of pretrained language models . |
| Outcome: | The proposed datasets show that they perform better than existing datasets in English . the proposed framework allows for sample-efficient few-shot transfer for TOD tasks . |
Zero and Few-Shot Localization of Task-Oriented Dialogue Agents with a Distilled Representation (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing low-cost approaches to build a high-quality functioning dialogue agent are limited to a few widely-spoken languages. |
| Approach: | They propose automatic methods that use ToD training data to build a functioning agent in another language . they compare the method to existing methods that only use a small training set . |
| Outcome: | The proposed method improves the state-of-the-art in Chinese to English transfer using zero-shot data compared to existing full-shot methods . the proposed method achieves 46.7% and 22.0% in task success rate and dialogue success rate, respectively. |
Cross-Lingual Dialogue Dataset Creation via Outline-Based Generation (2023.tacl-1)
Copied to clipboard
| Challenge: | Multilingual task-oriented dialogue (ToD) datasets suffer from severe limitations, such as being small in scale and lacking naturalness and cultural specificity in the target language. |
| Approach: | They propose a novel outline-based annotation process where domain-specific abstract schemata of dialogue are mapped into natural language outlines. |
| Outcome: | The proposed approach improves understanding, dialogue state tracking, and end-to-end dialogue evaluation in Arabic, Indonesian, Russian, and Kiswahili. |
MultiWOZ - A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling (D18-1)
Copied to clipboard
Paweł Budzianowski, Tsung-Hsien Wen, Bo-Hsiang Tseng, Iñigo Casanueva, Stefan Ultes, Osman Ramadan, Milica Gašić
| Challenge: | a dataset of 10k human-human written conversations is one order of magnitude larger than previous annotated task-oriented corpora. |
| Approach: | They propose to collect 10k human-human written conversations from a crowd-sourced dataset using crowd-sourcing. |
| Outcome: | The proposed dataset is one order of magnitude larger than previous annotated task-oriented corpora and shows the usability of the data and sets a baseline for future studies. |