| Challenge: | a number of questions contain questionable assumptions, such as when did Marie Curie discover Uranium, that cannot be answered as a true when question. |
| Approach: | They propose an open-domain evaluation dataset that can detect questionable assumptions . they propose a method that can be used to produce adequate responses for questions with questionable assumption. |
| Outcome: | The proposed model detects questionable assumptions and produces adequate responses for both types of questions. |
Similar Papers
Identifying and Answering Questions with False Assumptions: An Interpretable Approach (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) generate misleading answers because of hallucinations . despite their capabilities, LLMs suffer from hallucinisms, which leads to unfaithful answers . |
| Approach: | They propose a method to identify and answer questions with false assumptions . they first investigate whether the problem reduces to fact verification . then, they leverage external evidence to mitigate hallucinations . |
| Outcome: | The proposed approach reduces the problem to fact verification and provides interpretable answers by pinpointing the false assumptions. |
NoiseQA: Challenge Set Evaluation for User-Centric Question Answering (2021.eacl-main)
Copied to clipboard
| Challenge: | Question-Answering (QA) systems are deployed in the real world . a lack of research attention has been devoted to studying the issues that arise when people use QA systems. |
| Approach: | They show that component components that precede an answering engine can introduce varied and considerable sources of error. |
| Outcome: | The proposed evaluations highlight the need for QA evaluation to expand to consider real-world use. |
Can NLI Models Verify QA Systems’ Predictions? (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Recent question answering systems perform well on benchmark datasets, but are not always well-calibrated to spot spurious answers under distribution shifts. |
| Approach: | They propose to use natural language inference to verify whether answers are correct . they leverage large pre-trained models and recent prior datasets to construct powerful question conversion and decontextualization modules. |
| Outcome: | The proposed approach improves the confidence estimation of a QA model across different domains, evaluated in a selective QA setting. |
Latent Retrieval for Weakly Supervised Open Domain Question Answering (P19-1)
Copied to clipboard
| Challenge: | Recent work on open domain question answering (QA) assumes strong supervision of the supporting evidence and/or assumes a blackbox information retrieval (IR) system to retrieve evidence candidates. |
| Approach: | They propose to jointly learn the retriever and reader from question-answer string pairs and without any IR system. |
| Outcome: | The proposed approach outperforms BM25 on open datasets with a learner and reader by 19 points in exact match. |
Handling Anomalies of Synthetic Questions in Unsupervised Question Answering (2020.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to improve unsupervised Question Answering (UQA) are expensive and require additional datasets. |
| Approach: | They propose an unsupervised QA approach that generates QA training data automatically. |
| Outcome: | The proposed method improves unsupervised QA significantly across a number of QA tasks. |
CREPE: Open-Domain Question Answering with False Presuppositions (2023.acl-long)
Copied to clipboard
| Challenge: | Existing question answering datasets assume all questions have well defined answers. |
| Approach: | They propose a QA dataset containing a distribution of false presuppositions . they find that 25% of questions contain false presumptions . |
| Outcome: | The proposed model finds that 25% of questions contain false presuppositions . the model can find presuffpositions moderately well, but struggle when predicting correctness . |
No Questions are Stupid, but some are Poorly Posed: Understanding Poorly-Posed Information-Seeking Questions (2025.acl-long)
Copied to clipboard
| Challenge: | When a question is poorly posed, answerers struggle to converge on dominant interpretations, while models attempt comprehensive coverage by addressing many interpretations simultaneously. |
| Approach: | They propose a computational framework to study poorly-posedness of questions by generating spaces of potential interpretations and computing distributions based on interpretations chosen by answerers in the Reddit question thread. |
| Outcome: | The proposed framework analyzes poorly-posed questions using a set of interpretations chosen by human answerers and large language models. |
Evaluation Paradigms in Question Answering (2021.emnlp-main)
Copied to clipboard
| Challenge: | Despite substantial overlap, subtle but significant distinctions exert an outsize influence on research . one paradigm values creating more intelligent QA systems, the other paradigm values building QA system that appeals to users. |
| Approach: | They propose to use the Cranfield and Manchester paradigms to describe research working towards building human-like, intelligent QA systems. |
| Outcome: | The proposed paradigms are based on the findings of two recent studies on question answering (QA) the Cranfield paradigm is not new, but the Manchester paradigm is christened as the most eclectic in QA . |
Using contradictions improves question answering systems (2023.acl-short)
Copied to clipboard
| Challenge: | Existing systems that use contradiction to determine if a question is supported by background contexts do better than those that use entailment. |
| Approach: | They propose a method that incorporates contradiction in natural language inference (NLI) they propose to reformulate answers from QA systems as hypotheses and then select the best one based on the results. |
| Outcome: | The proposed method improves on multiple choice and extractive QA in two settings. |
Toward Deconfounding the Effect of Entity Demographics for Question Answering Accuracy (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing question answering datasets lack diversity in gender, profession, and nationality. |
| Approach: | They focus on how well QA models generalize across demographic subsets . english-language QA datasets mostly ask about US men from a few professions - this is problematic because most English speakers are not from the US or UK . |
| Outcome: | The proposed model accuracy is lower for people based on gender, profession, and nationality, but there is more variation on professions (question topic) and question ambiguity. |