| Challenge: | Existing non-task oriented dialogue systems can yield a relevant and fluent response, but sometimes make logical mistakes because of weak reasoning capabilities. |
| Approach: | They propose a dataset for multi-turn dialogue reasoning that uses annotated dialogues to train a machine to handle various reasoning problems. |
| Outcome: | Empirical results show that state-of-the-art methods only reach 71%, far behind human performance of 94%. |
Similar Papers
Natural Language Processing for Multilingual Task-Oriented Dialogue (2022.acl-tutorials)
Copied to clipboard
| Challenge: | a tutorial will examine the challenges and gaps in multilingual ToD research . multilingual systems are difficult to build, and are limited to English and other languages . |
| Approach: | This tutorial will discuss the importance of multilingual task-oriented dialogue systems . it will provide an overview of current research gaps, challenges and initiatives related to multilingual ToD systems - with a particular focus on their connections to current research and challenges in multilingual and low-resource NLP. |
| Outcome: | This tutorial will provide an overview of current research gaps, challenges and initiatives related to multilingual ToD systems. |
Dialog Generation Using Multi-Turn Reasoning Neural Networks (N18-1)
Copied to clipboard
| Challenge: | Existing methods for dialog generation are limited and short at generalization. |
| Approach: | They propose a generalizable dialog generation approach that adapts multi-turn reasoning to generate responses by taking current conversation session context as a document and current query as 'question' they separate the single memory used for document comprehension into different groups for speaker-specific topic and opinion embedding. |
| Outcome: | Experiments on Japanese 10-sentence (5-round) conversation modeling show that multi-turn reasoning can produce more diverse and acceptable responses than state-of-the-art single-turn and non-reasoning baselines. |
Cross-Lingual Dialogue Dataset Creation via Outline-Based Generation (2023.tacl-1)
Copied to clipboard
| Challenge: | Multilingual task-oriented dialogue (ToD) datasets suffer from severe limitations, such as being small in scale and lacking naturalness and cultural specificity in the target language. |
| Approach: | They propose a novel outline-based annotation process where domain-specific abstract schemata of dialogue are mapped into natural language outlines. |
| Outcome: | The proposed approach improves understanding, dialogue state tracking, and end-to-end dialogue evaluation in Arabic, Indonesian, Russian, and Kiswahili. |
To Chat or Task: a Multi-turn Dialogue Generation Framework for Task-Oriented Dialogue Systems (2025.acl-industry)
Copied to clipboard
| Challenge: | Large language models (LLMs) are designed to handle complex task requests, but lack of specific datasets for training and evaluation of such systems . |
| Approach: | They propose a framework to generate a dataset for in-vehicle speech recognition systems . they train an in-car context sensor that correctly identifies the functional intent of the driver . |
| Outcome: | The proposed framework outperforms baseline models across experimental settings. |
RECOR: Reasoning-focused Multi-turn Conversational Retrieval Benchmark (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks treat multi-turn conversation and reasoning-intensive retrieval separately, yet real-world information seeking requires both. |
| Approach: | They propose a framework that transforms complex queries into fact-grounded multi-turn dialogues through multi-level validation. |
| Outcome: | The proposed framework outperforms existing systems in a number of domains and can be used to improve multi-turn conversation retrieval. |
KdConv: A Chinese Multi-domain Dialogue Dataset Towards Multi-turn Knowledge-driven Conversation (2020.acl-main)
Copied to clipboard
| Challenge: | Existing knowledge-driven dialog data is limited due to the lack of dialog data which consists of multi-turn conversations on multiple topics and with knowledge annotations. |
| Approach: | They propose a Chinese multi-domain knowledge-driven conversation dataset which grounds the topics in multi-turn conversations to knowledge graphs. |
| Outcome: | The proposed dataset can be enhanced by introducing background knowledge, but there is still a large space for leveraging knowledge to model multi-turn conversations for further research. |
The JDDC Corpus: A Large-Scale Multi-Turn Chinese Dialogue Dataset for E-commerce Customer Service (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing datasets for human-like dialogue tasks are deficient due to the complexity of human conversations. |
| Approach: | They construct a large-scale Chinese E-commerce conversation corpus with 1 million dialogues, 20 million utterances, and 150 million words. |
| Outcome: | The proposed dataset includes 1 million multi-turn dialogues, 20 million utterances, and 150 million words. |
More Diverse Dialogue Datasets via Diversity-Informed Data Collection (2020.acl-main)
Copied to clipboard
| Challenge: | Existing approaches to generate conversational dialogue produce uninteresting, predictable responses. |
| Approach: | They propose a method to collect and determine more diverse data from conversational participants . they use dynamically computed corpus-level statistics to determine which conversational participant to collect data from . |
| Outcome: | The proposed method produces significantly more diverse data than baseline methods and better results on emotion classification and dialogue generation tasks. |
xDial-Eval: A Multilingual Open-Domain Dialogue Evaluation Benchmark (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Currently, human evaluation is the most reliable way to holistically judge the quality of the dialogue. |
| Approach: | They propose to use English dialogue evaluation metrics to generalize them to other languages. |
| Outcome: | The proposed metrics outperform OpenAI’s ChatGPT in terms of average Pearson correlations over all datasets and languages. |
Contextual Semantic Parsing for Multilingual Task-Oriented Dialogues (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing methods for predicting state of a conversation are limited to a few languages . a method that can be applied to other languages will benefit the large population of speakers of many other languages. |
| Approach: | They propose to automatically translate large-scale dialogue data sets in one language to produce an effective semantic parser for other languages using machine translation. |
| Outcome: | The proposed model reduces the compounding effect of translation errors without harming the accuracy in practice. |