| Challenge: | Existing clarification datasets with limited annotated examples do not address ambiguous phenomena. |
| Approach: | They propose a dataset that allows users to ask clarification questions using open-domain examples. |
| Outcome: | The proposed model achieves better performance than strong baselines and provides new challenges. |
Similar Papers
ClarQ: A large-scale and diverse dataset for Clarification Question Generation (2020.acl-main)
Copied to clipboard
| Challenge: | Existing datasets hinder development of large-scale models capable of generating and utilising clarification questions. |
| Approach: | They propose a bootstrapping framework that utilises a neural network architecture to classify clarification questions based on post-comment tuples extracted from stackexchange. |
| Outcome: | The proposed framework aims to increase the accuracy of the classifier and increase recall of clarification questions by applying it to question-answering tasks. |
Asking Clarification Questions to Handle Ambiguity in Open-Domain QA (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Ambiguous questions persist in open-domain question answering because formulating a precise question with a unique answer is often challenging. |
| Approach: | They propose to ask a clarification question where the user’s response will help identify the interpretation that best aligns with the user's intention. |
| Outcome: | The proposed approach achieves F1 of 61.3, 25.1, and 40.5 on the three tasks, demonstrating the need for further improvements while providing competitive baselines for future work. |
Answer-based Adversarial Training for Generating Clarification Questions (N19-1)
Copied to clipboard
| Challenge: | a goal of natural language processing is to develop techniques that enable machines to process naturally occurring language. |
| Approach: | They propose a model where hypothetical answers are latent variables that can guide the model into generating more useful clarification questions. |
| Outcome: | The proposed model outperforms retrieval-based models and ablations that exclude utility model and adversarial training on two datasets. |
Ask what’s missing and what’s useful: Improving Clarification Question Generation using Global Knowledge (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing models that generate clarification questions fail to identify useful information in contexts . human ability to generate fluent and relevant questions is important in reducing ambiguity . |
| Approach: | They propose a model that first identifies what is missing and then generates a question about it. |
| Outcome: | The proposed model outperforms baselines as judged by automatic metrics and humans. |
Building a Dataset for Automatically Learning to Detect Questions Requiring Clarification (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing work on question answering systems assumes all questions are intelligible and unambiguous . however, available datasets do not meet requirements for building commercial virtual assistants . |
| Approach: | They propose to make question answering systems more robust by classifying if question is intelligible and returning a clarification question for contextual ambiguity. |
| Outcome: | The proposed system can classify if the input question is intelligible and return a clarification question for ambiguous questions. |
ASK: Aspects and Retrieval based Hybrid Clarification in Task Oriented Dialogue Systems (2025.acl-industry)
Copied to clipboard
| Challenge: | Ambiguous user queries pose a challenge in task-oriented dialogue systems . Large Language Models (LLMs) rely on the top-k retrieved documents for clarification . traditional approaches lack principled mechanisms to determine when to use broad domain knowledge vs specific retrieved document context for clarification. |
| Approach: | They propose a hybrid approach that dynamically chooses between document-based or aspect-based clarification based on query ambiguity. |
| Outcome: | The proposed approach shows significant improvements over baselines on product troubleshooting and product search datasets. |
A Survey on Asking Clarification Questions Datasets in Conversational Systems (2023.acl-long)
Copied to clipboard
| Challenge: | Existing studies on Asking Clarification Questions (ACQs) are incomparable due to inconsistent data, experimental setups and evaluation strategies. |
| Approach: | They analyse the current research status on Asking Clarification Questions (ACQs) and propose a set of evaluation metrics and benchmarks for multiple ACQs-related tasks. |
| Outcome: | The proposed techniques are compared with the available datasets and evaluated against benchmarks. |
Tree of Clarifications: Answering Ambiguous Questions with Retrieval-Augmented Large Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | In open-domain question answering, users often ask ambiguous questions (AQs) . one approach is to identify all possible interpretations of the AQ and generate a long-form answer addressing them all. |
| Approach: | They propose a framework that generates a long-form answer addressing all possible interpretations of an ambiguous question. |
| Outcome: | The proposed framework outperforms baselines on ASQA in a few-shot setup across metrics while surpassing fully-supervised baselines trained on the whole training set in terms of Disambig-F1 and Disambigo-ROUGE. |
Python Code Generation by Asking Clarification Questions (2023.acl-long)
Copied to clipboard
| Challenge: | Recent work addresses text-to-code generation using pretrained language models (PLMs) for large-scale NLD: Logistic Regression. |
| Approach: | They propose a dataset containing pairs of natural language descriptions and code with created synthetic clarification questions and answers to solve the under-specified nature of a natural language description. |
| Outcome: | The proposed model improves on previous models, while introducing new challenges to the community, including when and what clarification questions should be asked. |
Learning to Ask Good Questions: Ranking Clarification Questions using Neural Expected Value of Perfect Information (P18-1)
Copied to clipboard
| Challenge: | StackExchange users routinely ask clarifying questions to fill information gaps . a principle goal of asking questions is to fill this information gap . |
| Approach: | They build a model to rank candidates by their usefulness to a given post . they use data from StackExchange to evaluate the model against human judgments . |
| Outcome: | The proposed model outperforms baselines on 500 samples of StackExchange's clarification questions. |