ConditionalQA: A Complex Reading Comprehension Dataset with Conditional Answers (2022.acl-long)
Copied to clipboard
| Challenge: | Existing datasets for reading comprehension have deterministic answers, but questions in the real world do not always have definite answers. |
| Approach: | They propose a Question Answering (QA) dataset that contains complex questions with conditional answers. |
| Outcome: | The proposed dataset will motivate further research in answering complex questions over long documents. |
Similar Papers
MultiSpanQA: A Dataset for Multi-Span Question Answering (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing reading comprehension datasets focus on single-span answers, but multi-spread questions are less studied. |
| Approach: | They propose a new reading comprehension dataset that focuses on multi-span questions . they introduce new metrics for the purposes of multi--spontaneous question answering evaluation . |
| Outcome: | The proposed model beats baselines and achieves state-of-the-art on the existing dataset. |
CMQA: A Dataset of Conditional Question Answering with Multiple-Span Answers (2022.coling-1)
Copied to clipboard
| Challenge: | Existing QA datasets only contain unconditional and parallel answers . conditional question answering with hierarchical multi-span answers is challenging for the community to solve . |
| Approach: | They propose a conditional question answering task with hierarchical multi-span answers . they propose CMQA, which contains conditional and hierarchic samples . |
| Outcome: | The proposed task can be used to build more reliable and sophisticated QA systems. |
MDCR: A Dataset for Multi-Document Conditional Reasoning (2024.findings-emnlp)
Copied to clipboard
| Challenge: | ConditionalQA is limited to questions on single documents, neglecting harder cases that may require *cross-document reasoning* and *optimization*. |
| Approach: | They propose to use a dataset to evaluate models' ability to answer eligibility questions on single documents. |
| Outcome: | The proposed dataset can reflect real-world challenges and serve as a test bed for complex conditional reasoning that requires optimization. |
HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering (D18-1)
Copied to clipboard
Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, Christopher D. Manning
| Challenge: | Existing question answering (QA) datasets fail to train QA systems to perform complex reasoning and provide explanations for answers. |
| Approach: | They propose a new dataset with 113k Wikipedia-based question-answer pairs with four key features: (1) the questions require finding and reasoning over multiple supporting documents to answer; (2) the questions are diverse and not constrained to any pre-existing knowledge bases or knowledge schemas; (3) the questions provide sentence-level supporting facts required for reasoning; and (4) a type of factoid comparison questions to test QA systems’ ability to extract relevant facts and perform necessary comparison. |
| Outcome: | The proposed dataset has 113k Wikipedia-based question-answer pairs and four key features that make it challenging for the latest QA systems. |
KQA Pro: A Dataset with Explicit Compositional Programs for Complex Question Answering over Knowledge Base (2022.acl-long)
Copied to clipboard
Shulin Cao, Jiaxin Shi, Liangming Pan, Lunyiu Nie, Yutong Xiang, Lei Hou, Juanzi Li, Bin He, Hanwang Zhang
| Challenge: | Existing benchmarks for Complex KBQA lack compositional reasoning capabilities . Existing methods for Complex questions are poor in diversity or scale . |
| Approach: | They propose a compositional programming language to represent the reasoning process of complex questions. |
| Outcome: | The proposed dataset includes around 120K diverse natural language questions . it provides a compositional and interpretable programming language to represent the reasoning process of complex questions based on the proposed model . |
DBQR-QA: A Question Answering Dataset on a Hybrid of Database Querying and Reasoning (2024.findings-acl)
Copied to clipboard
| Challenge: | Question answering (QA) is a fundamental task in the field of Natural Language Processing (NLP). |
| Approach: | They propose a database querying and reasoning dataset for question answering that is designed to accommodate sequential questions and multi-hop queries. |
| Outcome: | The proposed dataset better mirrors the dynamics of real-world information retrieval and analysis with a particular focus on the financial reports of US companies. |
Discourse Comprehension: A Question Answering Framework to Represent Sentence Connections (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing systems for text comprehension are inadequate for more holistic comprehension of a discourse. |
| Approach: | They propose a new paradigm that captures both discourse and semantic links between sentences in the form of free-form, open-ended questions. |
| Outcome: | The proposed model captures discourse and semantic links between sentences in the form of free-form, open-ended questions. |
SciDQA: A Deep Reading Comprehension Dataset over Scientific Papers (2024.emnlp-main)
Copied to clipboard
| Challenge: | SciDQA is a dataset for question-answering that challenges language models to deeply understand scientific articles. |
| Approach: | They propose a new dataset for reading comprehension that challenges language models to deeply understand scientific articles consisting of 2,937 QA pairs. |
| Outcome: | The SciDQA dataset is based on 2,937 QA pairs and decontextualizes the content, tracks the source document across different versions, and incorporates a bibliography for multi-document question-answering. |
IfQA: A Dataset for Open-domain Question Answering under Counterfactual Presuppositions (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing open-domain QA tasks focus on questions whose answer can be deduced directly from global factual knowledge. |
| Approach: | They propose a dataset where each question is based on a counterfactual presupposition via an "if" clause. |
| Outcome: | The IfQA dataset contains 3,800 questions that were annotated by crowdworkers on relevant Wikipedia passages. |
ComQA: A Community-sourced Dataset for Complex Factoid Question Answering with Paraphrase Clusters (N19-1)
Copied to clipboard
| Challenge: | ComQA dataset captures question phenomena and the diverse ways in which they are formulated. |
| Approach: | They propose a large dataset of real user questions that captures question phenomena and the diverse ways in which they are formulated. |
| Outcome: | The proposed dataset can be a driver of future research on factoid question answering (QA). |