PhotoshopQuiA: A Corpus of Non-Factoid Questions and Answers for Why-Question Answering (L18-1)
Copied to clipboard
| Challenge: | Community Question Answering web sites are used for non-factoid question answering . however, there is a scarcity of available datasets for this task . cnn.com's john m. sutter is releasing a dataset for why-QA . |
| Approach: | They propose a dataset of 2,854 why-question and answer(s) pairs related to Adobe Photoshop usage from five CQA web sites. |
| Outcome: | The new dataset is the first English dataset for Why-QA that focuses on a product . it can be used to build Why-Q systems, evaluate approaches and develop new models . |
Similar Papers
ComQA: A Community-sourced Dataset for Complex Factoid Question Answering with Paraphrase Clusters (N19-1)
Copied to clipboard
| Challenge: | ComQA dataset captures question phenomena and the diverse ways in which they are formulated. |
| Approach: | They propose a large dataset of real user questions that captures question phenomena and the diverse ways in which they are formulated. |
| Outcome: | The proposed dataset can be a driver of future research on factoid question answering (QA). |
CCQA: A New Web-Scale Question Answering Dataset for Model Pre-Training (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing approaches to answer open domain questions rely on unlabeled text or synthetically generated question-answer pairs. |
| Approach: | They propose a large-scale open-domain question-answering dataset based on the Common Crawl project that can be used to in-domain pre-train popular language models. |
| Outcome: | The proposed dataset achieves promising results in zero-shot, low resource and fine-tuned settings across multiple tasks, models and benchmarks. |
JDocQA: Japanese Document Question Answering Dataset for Generative Language Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Document question answering is a task of question answering on given documents such as reports, slides, pamphlets, and websites. |
| Approach: | They propose a large-scale document-based QA dataset that requires both visual and textual information to answer questions. |
| Outcome: | The proposed dataset incorporates multiple categories of questions and unanswerable questions from the document for realistic question-answering applications. |
NeuralQA: A Usable Library for Question Answering (Contextual Query Expansion + BERT) on Large Datasets (2020.emnlp-demos)
Copied to clipboard
| Challenge: | Existing tools for Question Answering (QA) have challenges that limit their use in practice. |
| Approach: | They propose a library that integrates with existing infrastructure and offers helpful defaults for QA subtasks. |
| Outcome: | NeuralQA integrates well with existing infrastructure and offers helpful defaults for QA subtasks. |
ELQA: A Corpus of Metalinguistic Questions and Answers about English (2023.acl-long)
Copied to clipboard
| Challenge: | ELQA corpus is metalinguistic—it consists of language about language. |
| Approach: | They present a corpus of questions and answers in and about the English language . they use a free-form question answering task and multiple LLMs to analyze their capacity . |
| Outcome: | The ELQA corpus covers grammar, meaning, fluency, and etymology . the results can be used to investigate metalinguistic capabilities of NLU models . |
CausalQA: A Benchmark for Causal Question Answering (2022.coling-1)
Copied to clipboard
Alexander Bondarenko, Magdalena Wolska, Stefan Heindorf, Lukas Blübaum, Axel-Cyrille Ngonga Ngomo, Benno Stein, Pavel Braslavski, Matthias Hagen, Martin Potthast
| Challenge: | Existing causal question answering datasets are relatively small and only include one type of causal question. |
| Approach: | They construct a benchmark corpus of 1.1 million causal questions with answers . they use a typology derived from a data-driven, manual analysis of QA datasets . |
| Outcome: | The proposed model achieves a ROUGE-L F1 score of 0.48 on the new QA benchmark. |
AraVQA: Building a New Arabic Factoid Visual Question Answering Dataset from Wikipedia (2026.acl-long)
Copied to clipboard
| Challenge: | Existing Arabic VQA datasets focus on culturally-specific and dialect-aware domains. |
| Approach: | They propose a pipeline that leverages Wikipedia template tags to extract relevant information for each image and utilize it to generate a new visual question answering dataset. |
| Outcome: | The proposed pipeline can enhance existing VLMs on Arabic VQA tasks by leveraging Wikipedia template tags. |
ConditionalQA: A Complex Reading Comprehension Dataset with Conditional Answers (2022.acl-long)
Copied to clipboard
| Challenge: | Existing datasets for reading comprehension have deterministic answers, but questions in the real world do not always have definite answers. |
| Approach: | They propose a Question Answering (QA) dataset that contains complex questions with conditional answers. |
| Outcome: | The proposed dataset will motivate further research in answering complex questions over long documents. |
Learning to Explain: Datasets and Models for Identifying Valid Reasoning Chains in Multihop Question-Answering (2020.emnlp-main)
Copied to clipboard
| Challenge: | despite rapid progress in multihop question-answering, models still have trouble explaining why an answer is correct. |
| Approach: | They propose three explanation datasets in which explanations from corpus facts are annotated . they first annotate multiple candidate explanations for each answer, then use crowd-sourcing perturbations to test generalization . |
| Outcome: | The proposed datasets improve explanation quality but still behind the upper bound . the proposed dataset can be used to improve explanations using a BERT-based classifier . |
WikiHowQA: A Comprehensive Benchmark for Multi-Document Non-Factoid Question Answering (2023.acl-long)
Copied to clipboard
| Challenge: | Answering non-factoid questions (NFQs) is a challenging task, requiring passage-level answers that are difficult to construct and evaluate. |
| Approach: | They propose a multi-document NFQA benchmark built on WikiHow, a website dedicated to answering “how-to” questions. |
| Outcome: | The proposed framework includes 11,746 human-written answers along with 74,527 supporting documents. |