| Challenge: | In response to this development, there have been a flurry of new datasets for question answering. |
| Approach: | They propose to use SQuAD 2.0, QuAC, and CoQA to provide question answering on textual data. |
| Outcome: | The proposed datasets provide complementary coverage of the first two aspects, but weak coverage of third. |
Similar Papers
QuAC: Question Answering in Context (D18-1)
Copied to clipboard
Eunsol Choi, He He, Mohit Iyyer, Mark Yatskar, Wen-tau Yih, Yejin Choi, Percy Liang, Luke Zettlemoyer
| Challenge: | a dataset for Question Answering in Context contains 14K information-seeking QA dialogs . questions are often more open-ended, unanswerable, or only meaningful within the dialog context . |
| Approach: | They propose a dataset for Question Answering in Context that contains 14K dialogs . they use a student to ask questions about a Wikipedia section and a teacher to answer them . |
| Outcome: | The proposed dataset underperforms humans in a number of reference models . the dataset contains 14K information-seeking dialogs over sections from Wikipedia . |
CoQAR: Question Rewriting on CoQA (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing systems that ask questions in a conversational context may have contextual dependencies that make the understanding difficult. |
| Approach: | They propose to rewrite questions into an out-of-context form to facilitate understanding . they propose to use this form to train and evaluate conversational question answering models . |
| Outcome: | The proposed model can be used in the supervised learning of three tasks: question paraphrasing, question rewriting and conversational question answering. |
MESAQA: A Dataset for Multi-Span Contextual and Evidence-Grounded Question Answering (2025.coling-main)
Copied to clipboard
| Challenge: | Existing question answering systems focus on extracting answers from single spans, but real-world scenarios require synthesizing information from multiple spans. |
| Approach: | They propose a dataset that leverages the MASH-QA dataset and large language models (LLMs) to ensure that each Q/A pair requires considering all selected spans. |
| Outcome: | The proposed method enables the model to answer multiple Q/A pairs in a single span, while ensuring that all selected spans are considered. |
XQA: A Cross-lingual Open-domain Question Answering Dataset (P19-1)
Copied to clipboard
| Challenge: | Open-domain question answering aims to answer questions through text retrieval and reading comprehension . but, the success of these models relies on a massive volume of training data, which is not available in other languages . a new dataset aims at investigating cross-lingual OpenQA . |
| Approach: | They propose to use a dataset for cross-lingual OpenQA research to test models . they use XQA dataset to train models with large volumes of labeled data . |
| Outcome: | The proposed model achieves best results in almost all target languages while the performance is lower than that of English. |
UNIFIEDQA: Crossing Format Boundaries with a Single QA System (2020.findings-emnlp)
Copied to clipboard
Daniel Khashabi, Sewon Min, Tushar Khot, Ashish Sabharwal, Oyvind Tafjord, Peter Clark, Hannaneh Hajishirzi
| Challenge: | Question answering (QA) tasks have been posed using a variety of formats . a new study aims to develop specialized QA models that can be used to train QA systems . |
| Approach: | They build a pre-trained question answering model that performs well across 19 QA datasets . they argue that format-specialized models can limit the ability to teach reasoning . |
| Outcome: | a new model that trains on QA datasets performs on par with 8 models trained on individual datasets . a single model that trained on UNIFIEDQA performs well on 19 QA data . |
Harvesting and Refining Question-Answer Pairs for Unsupervised QA (2020.acl-main)
Copied to clipboard
| Challenge: | Recent research attempts to extend unsupervised question answering to settings with few or no labeled data available. |
| Approach: | They propose two approaches to improve unsupervised question answering . first, they harvest lexically and syntactically divergent Wikipedia questions to automatically construct a corpus of question-answer pairs . second, they take advantage of the QA model to extract more appropriate answers . |
| Outcome: | The proposed approach outperforms previous unsupervised approaches by a large margin and is competitive with early supervised models. |
Know What You Don’t Know: Unanswerable Questions for SQuAD (P18-2)
Copied to clipboard
| Challenge: | Existing datasets focus on answerable questions or use automatically generated unanswerable questions that are easy to identify. |
| Approach: | They propose a dataset that combines the Stanford Question Answering Dataset with 50,000 unanswerable questions written by crowdworkers to look similar to answerable ones. |
| Outcome: | The proposed dataset looks similar to answerable questions on crowd-written questions . strong neural system that gets 86% F1 on SQuAD achieves only 66% F1. |
UQA: Corpus for Urdu Question Answering (2024.lrec-main)
Copied to clipboard
| Challenge: | Urdu is a low-resource language with over 70 million native speakers . expanding the reach of NLP to languages other than English is crucial for advancing multilingual AI systems. |
| Approach: | They introduce a novel dataset for question answering and text comprehension in Urdu . they use a technique called EATS which preserves the answer spans in translated context paragraphs . |
| Outcome: | The proposed dataset preserves answer spans in translated context paragraphs. |
CCQA: A New Web-Scale Question Answering Dataset for Model Pre-Training (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing approaches to answer open domain questions rely on unlabeled text or synthetically generated question-answer pairs. |
| Approach: | They propose a large-scale open-domain question-answering dataset based on the Common Crawl project that can be used to in-domain pre-train popular language models. |
| Outcome: | The proposed dataset achieves promising results in zero-shot, low resource and fine-tuned settings across multiple tasks, models and benchmarks. |
ReQA: An Evaluation for End-to-End Answer Retrieval Models (D19-58)
Copied to clipboard
| Challenge: | Popular QA benchmarks like SQuAD have driven progress on identifying answer spans within a specific passage . retrieving relevant answers from a huge corpus of documents is still a challenging problem . |
| Approach: | They propose a benchmark for evaluating large-scale sentence-level answer retrieval models . they establish baselines using both neural encoding models and classical retrieval techniques . |
| Outcome: | The proposed model outperforms human models on identifying answer spans within a specific passage . the proposed model is scalable and can bypass the typical document retrieval step . |