Asking Clarification Questions to Handle Ambiguity in Open-Domain QA (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Ambiguous questions persist in open-domain question answering because formulating a precise question with a unique answer is often challenging. |
| Approach: | They propose to ask a clarification question where the user’s response will help identify the interpretation that best aligns with the user's intention. |
| Outcome: | The proposed approach achieves F1 of 61.3, 25.1, and 40.5 on the three tasks, demonstrating the need for further improvements while providing competitive baselines for future work. |
Similar Papers
Tree of Clarifications: Answering Ambiguous Questions with Retrieval-Augmented Large Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | In open-domain question answering, users often ask ambiguous questions (AQs) . one approach is to identify all possible interpretations of the AQ and generate a long-form answer addressing them all. |
| Approach: | They propose a framework that generates a long-form answer addressing all possible interpretations of an ambiguous question. |
| Outcome: | The proposed framework outperforms baselines on ASQA in a few-shot setup across metrics while surpassing fully-supervised baselines trained on the whole training set in terms of Disambig-F1 and Disambigo-ROUGE. |
AmbigQA: Answering Ambiguous Open-domain Questions (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing open-domain question answering systems assume questions have a single welldefined answer. |
| Approach: | They propose an open-domain question answering task which involves finding every plausible answer and rewriting the question for each one to resolve the ambiguity. |
| Outcome: | The proposed task is based on a dataset covering 14,042 open-domain questions . it shows that strong models benefit from weakly supervised learning . |
Model Analysis & Evaluation for Ambiguous Question Answering (2023.findings-acl)
Copied to clipboard
| Challenge: | Ambiguous questions are a challenge for Question Answering models as they require answers that cover multiple interpretations of the original query. |
| Approach: | They aim to investigate whether model/data scaling improves the answers’ quality and whether automated metrics align with human judgment. |
| Outcome: | The proposed models can generate long-form answers that combine conflicting information and provide valuable insights into the limitations of the current approaches. |
Asking Clarification Questions in Knowledge-Based Question Answering (D19-1)
Copied to clipboard
| Challenge: | Existing clarification datasets with limited annotated examples do not address ambiguous phenomena. |
| Approach: | They propose a dataset that allows users to ask clarification questions using open-domain examples. |
| Outcome: | The proposed model achieves better performance than strong baselines and provides new challenges. |
ASK: Aspects and Retrieval based Hybrid Clarification in Task Oriented Dialogue Systems (2025.acl-industry)
Copied to clipboard
| Challenge: | Ambiguous user queries pose a challenge in task-oriented dialogue systems . Large Language Models (LLMs) rely on the top-k retrieved documents for clarification . traditional approaches lack principled mechanisms to determine when to use broad domain knowledge vs specific retrieved document context for clarification. |
| Approach: | They propose a hybrid approach that dynamically chooses between document-based or aspect-based clarification based on query ambiguity. |
| Outcome: | The proposed approach shows significant improvements over baselines on product troubleshooting and product search datasets. |
Answering Ambiguous Questions through Generative Evidence Fusion and Round-Trip Prediction (2021.acl-long)
Copied to clipboard
Yifan Gao, Henghui Zhu, Patrick Ng, Cicero Nogueira dos Santos, Zhiguo Wang, Feng Nan, Dejiao Zhang, Ramesh Nallapati, Andrew O. Arnold, Bing Xiang
| Challenge: | Open-domain question answering is a task to answer questions using passages with diverse topics. |
| Approach: | They propose a model that aggregates evidence from multiple passages to adaptively predict a single answer or a set of question-answer pairs for ambiguous questions. |
| Outcome: | The proposed model achieves state-of-the-art performance on AmbigQA dataset and shows competitive performance on NQ-Open and TriviaQA. |
Teaching Vision-Language Models to Ask: Resolving Ambiguity in Visual Questions (2025.acl-long)
Copied to clipboard
| Challenge: | Existing research addresses ambiguous visual questions by rephrasing questions, but it fails to address the inherently interactive nature of user interactions with visual language models (VLMs). Existing studies focus on re-phrase questions, and lack of a benchmark to assess VLMs’ capacity for resolving ambiguities through interaction. |
| Approach: | They propose a visual question answering task that provides a natural language answer to a question based on a given image and an automated pipeline to generate ambiguity-clarification question pairs. |
| Outcome: | The proposed benchmark targets three common categories of ambiguity in visual question answering (VQA) context and encompasses various VQA scenarios. |
Answering Ambiguous Questions via Iterative Prompting (2023.acl-long)
Copied to clipboard
| Challenge: | Empirical studies show that AmbigPrompt achieves state-of-the-art or competitive results while using less memory and having a lower inference latency than competing approaches. |
| Approach: | They propose an answering model with a prompting model to address imperfections in open-domain question answering . Empirical studies show AmbigPrompt achieves state-of-the-art or competitive results . |
| Outcome: | The proposed framework improves on two commonly-used open benchmarks and achieves state-of-the-art or competitive results while using less memory and having a lower inference latency. |
Building and Evaluating Open-Domain Dialogue Corpora with Clarifying Questions (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances on neural approaches to natural language processing have triggered a renaissance in end-to-end neural open-domain chatbots. |
| Approach: | They propose to use offline and online steps to evaluate the quality of clarifying questions in various open-domain dialogues to improve the quality and accuracy of the system response. |
| Outcome: | The proposed pipeline is suitable as a foundation for further research. |
Detecting Temporal Ambiguity in Questions (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Ambiguous questions have different answers depending on their interpretation and can take diverse forms. |
| Approach: | They propose a manually annotated temporally ambiguous QA dataset that captures temporal ambiguity and propose different search strategies based on disambiguate versions of the questions. |
| Outcome: | The proposed approach captures temporal ambiguity and provides non-search, competitive baselines for detecting temporal and few-shot ambiguities. |