FreebaseQA: A New Factoid QA Data Set Matching Trivia-Style Question-Answer Pairs with Freebase (N19-1)
Copied to clipboard
| Challenge: | Using FreebaseQA, we can generate over 54K matches from about 28K unique questions with minimal cost. |
| Approach: | They propose a data set for open-domain factoid question answering tasks over structured knowledge bases, like Freebase, using a combination of trivia-type question-answer pairs and subject-predicate-object triples. |
| Outcome: | The proposed data set generates 54K matches from 28K unique questions with minimal cost. |
Similar Papers
ComQA: A Community-sourced Dataset for Complex Factoid Question Answering with Paraphrase Clusters (N19-1)
Copied to clipboard
| Challenge: | ComQA dataset captures question phenomena and the diverse ways in which they are formulated. |
| Approach: | They propose a large dataset of real user questions that captures question phenomena and the diverse ways in which they are formulated. |
| Outcome: | The proposed dataset can be a driver of future research on factoid question answering (QA). |
HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering (D18-1)
Copied to clipboard
Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, Christopher D. Manning
| Challenge: | Existing question answering (QA) datasets fail to train QA systems to perform complex reasoning and provide explanations for answers. |
| Approach: | They propose a new dataset with 113k Wikipedia-based question-answer pairs with four key features: (1) the questions require finding and reasoning over multiple supporting documents to answer; (2) the questions are diverse and not constrained to any pre-existing knowledge bases or knowledge schemas; (3) the questions provide sentence-level supporting facts required for reasoning; and (4) a type of factoid comparison questions to test QA systems’ ability to extract relevant facts and perform necessary comparison. |
| Outcome: | The proposed dataset has 113k Wikipedia-based question-answer pairs and four key features that make it challenging for the latest QA systems. |
Farewell Freebase: Migrating the SimpleQuestions Dataset to DBpedia (C18-1)
Copied to clipboard
| Challenge: | Existing datasets for question answering over knowledge graphs lack answer triples from Freebase . a defunct knowledge graph makes it difficult to build "real-world" question answering systems . |
| Approach: | They propose a benchmark dataset for simple question answering over knowledge graphs that maps SimpleQuestions entities and predicates from Freebase to DBpedia. |
| Outcome: | The proposed dataset provides simple yet strong baselines with and without neural networks. |
FeTaQA: Free-form Table Question Answering (2022.tacl-1)
Copied to clipboard
Linyong Nan, Chiachun Hsieh, Ziming Mao, Xi Victoria Lin, Neha Verma, Rui Zhang, Wojciech Kryściński, Hailey Schoelkopf, Riley Kong, Xiangru Tang, Mutethia Mutuma, Ben Rosand, Isabel Trindade, Renusree Bandaru, Jacob Cunningham, Caiming Xiong, Dragomir Radev, Dragomir Radev
| Challenge: | Existing table-based question answering datasets lack advanced information-based questions that require reasoning and integration of information pieces retrieved from structured knowledge sources. |
| Approach: | They propose a dataset with 10K Wikipedia-based table, question, free-form answer, supporting table cells pairs that can be used to generate an answer. |
| Outcome: | The proposed dataset has 10K Wikipedia-based table, question, free-form answer, supporting table cells pairs. |
Explanations for CommonsenseQA: New Dataset and Models (2021.acl-long)
Copied to clipboard
Shourya Aggarwal, Divyanshu Mandowara, Vishwajeet Agrawal, Dinesh Khandelwal, Parag Singla, Dinesh Garg
| Challenge: | a dataset called CommonsenseQA (CQA) was recently released to advance the research on common-sense question answering (QA) |
| Approach: | They propose to retrieve and generate explanations for a given question, correct answer choice, incorrect answer choices tuple from a dataset called CommonsenseQA. |
| Outcome: | The proposed model beats baseline model by 100% in F1 score and similarity score of 61.9 . |
ASQA: Factoid Questions Meet Long-Form Answers (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent progress on factoid question answering (QA) does not easily transfer to the task of long-form QA where the goal is to generate detailed explanations. |
| Approach: | They propose a task that focuses on ambiguous factoid questions which have different correct answers depending on interpretation. |
| Outcome: | The proposed metric is reliable and demonstrates agreement between this metric and human judgments, and reveals a considerable gap between human performance and strong baselines. |
Augmenting Pre-trained Language Models with QA-Memory for Open-Domain Question Answering (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing methods for open-domain question-answering use an open book approach . a recent alternative is to retrieve from a collection of previously-generated question-annwer pairs . |
| Approach: | They propose a new QA system that augments a text-to-text model with a large memory of question-answer pairs and a task for the latent step of question retrieval. |
| Outcome: | The proposed system outperforms closed-book QA and can answer multi-hop questions. |
MESAQA: A Dataset for Multi-Span Contextual and Evidence-Grounded Question Answering (2025.coling-main)
Copied to clipboard
| Challenge: | Existing question answering systems focus on extracting answers from single spans, but real-world scenarios require synthesizing information from multiple spans. |
| Approach: | They propose a dataset that leverages the MASH-QA dataset and large language models (LLMs) to ensure that each Q/A pair requires considering all selected spans. |
| Outcome: | The proposed method enables the model to answer multiple Q/A pairs in a single span, while ensuring that all selected spans are considered. |
WikiHowQA: A Comprehensive Benchmark for Multi-Document Non-Factoid Question Answering (2023.acl-long)
Copied to clipboard
| Challenge: | Answering non-factoid questions (NFQs) is a challenging task, requiring passage-level answers that are difficult to construct and evaluate. |
| Approach: | They propose a multi-document NFQA benchmark built on WikiHow, a website dedicated to answering “how-to” questions. |
| Outcome: | The proposed framework includes 11,746 human-written answers along with 74,527 supporting documents. |
QUADRo: Dataset and Models for QUestion-Answer Database Retrieval (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing tools do not consider answers (question-question similarity only) or their quality in the annotation process. |
| Approach: | They propose an open-domain annotated resource that retrieves questions from pre-computed databases . they use a binary label associated with each question/answer pair to determine relevance . |
| Outcome: | The proposed model trains and evaluates models on 443,000 examples with respect to key aspects of QA systems. |