| Challenge: | Humans produce and interpret complex utterances even in simple scenarios. |
| Approach: | They present a large-scale English language corpus with 34,268 (polar question, indirect answer) pairs to enable progress on this task. |
| Outcome: | The proposed corpus contains 34,268 (polar question, indirect answer) pairs, and reaches 82-88% accuracy for a 4-class distinction, and 64-85% for 6 classes. |
Similar Papers
DIRECT: Direct and Indirect Responses in Conversational Text Corpus (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Neural conversation models have been able to generate fluent responses through training on a dialogue corpus, but they lack the ability to reveal the implied intentions of users. |
| Approach: | They propose to train neural conversation models on a dialogue corpus that provides pragmatic paraphrases to advance techniques for natural language understanding in dialogue systems. |
| Outcome: | The proposed corpus provides 71,498 pairs of indirect–direct utterance pairs accompanied by a multi-turn dialogue history extracted from the MultiWoZ dataset. |
Interpreting Indirect Answers to Yes-No Questions in Multiple Languages (2023.findings-emnlp)
Copied to clipboard
Zijie Wang, Md Hossain, Shivam Mathur, Terry Melo, Kadir Ozler, Keun Park, Jacob Quintero, MohammadHossein Rezaei, Shreya Shakya, Md Uddin, Eduardo Blanco
| Challenge: | Existing models for Yes-no questions skip polar keywords and instead use long explanations that must be interpreted. |
| Approach: | They propose a distant supervision approach to collect training data and show that direct answers are useful to train models to interpret indirect answers. |
| Outcome: | The proposed model achieves a 68% to 76% F1-score on multilingual Question-Answering benchmarks. |
IndirectQA: Understanding Indirect Answers to Implicit Polar Questions in French and Spanish (2024.lrec-main)
Copied to clipboard
| Challenge: | polar questions are common in spoken dialogues and expect exactly one of two answers (yes/no) but conversational systems struggle to interpret them. |
| Approach: | They propose to use subtitle data to interpret indirect answers in french and spanish . they use subtitles to broaden polar questions to include also implicit polar ones . |
| Outcome: | The proposed corpus of indirect answers shows that the task is challenging and challenging . the baseline accuracy scores drop from 61.43 on english to 44.06 for french and Spanish . |
Disentangling Indirect Answers to Yes-No Questions in Real Conversations (2022.naacl-main)
Copied to clipboard
Krishna Sanagavarapu, Jathin Singaraju, Anusha Kakileti, Anirudh Kaza, Aaron Mathews, Helen Li, Nathan Brito, Eduardo Blanco
| Challenge: | Existing models with synthetic indirect answers to yes-no questions are not beneficial when working with real conversations. |
| Approach: | They propose to annotate the underlying direct answers to yes-no questions in real conversations. |
| Outcome: | The proposed model outperforms the majority baseline but the task remains a challenge. |
Making Task-Oriented Dialogue Datasets More Natural by Synthetically Generating Indirect User Requests (2025.coling-main)
Copied to clipboard
| Challenge: | Existing task-oriented dialogue benchmarks lack sufficient examples of complex discourse phenomena such as indirectness. |
| Approach: | They propose a set of linguistic criteria and an LLM-based pipeline for generating realistic IURs to test natural language understanding and dialogue state tracking models before deployment in a new domain. |
| Outcome: | The proposed model can handle indirect user requests (IURs) but lacks examples of complex discourse phenomena such as indirectness. |
Interpreting Answers to Yes-No Questions in Dialogues from Multiple Domains (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing models for yes-no questions are challenging, but they still face challenges. |
| Approach: | They propose an approach grounded on distant supervision and blended training to quickly adapt to a new dialogue domain. |
| Outcome: | The proposed approach improves F1 performance in movie scripts, tennis interviews, and airline customer service domains. |
Unveiling the Limits of Large Language Models in Inferring Pragmatic Meaning from Non-Verbal Responses (2026.acl-long)
Copied to clipboard
| Challenge: | Existing studies have focused mainly on LLMs' comprehension of verbal behavior, with non-verbal behavior considered only in conjunction with verbal responses. |
| Approach: | They present the first systematic evaluation of LLMs’ ability to infer pragmatic meaning in dialogue consisting solely of non-verbal responses. |
| Outcome: | The proposed model fails to capture non-verbal intent and has accuracy dropping by 60% compared to verbal ones. |
“Mama Always Had a Way of Explaining Things So I Could Understand”: A Dialogue Corpus for Learning to Construct Explanations (2022.coling-1)
Copied to clipboard
| Challenge: | a new corpus of dialogical explanations is created to help explainable AI . a linguistic analysis of 65 transcribed English dialogues shows that explanations are co-constructed in a dialogue between the explainer and the explainee . |
| Approach: | They propose a corpus of dialogical explanations that are co-constructed in a dialogue . they analyze linguistic patterns of explainers and explainees and explore differences . |
| Outcome: | The proposed corpus of dialogical explanations enables NLP research on how humans explain . the analysis shows that sequence information helps predicting topics, acts, and moves effectively . |
PragmatiCQA: A Dataset for Pragmatic Question Answering in Conversations (2023.findings-acl)
Copied to clipboard
| Challenge: | Mars? - PragmatiCQA |
| Approach: | Mars? - The Paper . |
| Outcome: | The proposed dataset features 6873 QA pairs that explores pragmatic reasoning in conversations over a diverse set of topics. |
Pragmatics in Language Grounding: Phenomena, Tasks, and Modeling Approaches (2023.findings-emnlp)
Copied to clipboard
| Challenge: | People rely heavily on context to enrich meaning beyond what is literally said. |
| Approach: | They analyze how task goals, environmental contexts, and communicative affordances in each work enrich linguistic meaning. |
| Outcome: | The proposed frameworks are based on linguistic goals, environmental contexts, and communicative affordances to enrich linguistic meaning. |