PAQ: 65 Million Probably-Asked Questions and What You Can Do With Them (2021.tacl-1)
Copied to clipboard
Patrick Lewis, Yuxiang Wu, Linqing Liu, Pasquale Minervini, Heinrich Küttler, Aleksandra Piktus, Pontus Stenetorp, Sebastian Riedel
| Challenge: | Open-domain Question Answering models that directly leverage question-answer (QA) pairs show promise in terms of speed and memory compared with conventional models which retrieve and read from text corpora. |
| Approach: | They propose a question-answer (QA)-pair retriever to facilitate improved QA-patch models by introducing Probably Asked Questions (PAQ) they propose QA pair retriever, RePAQ, which preempts and caches test questions, enabling it to match the accuracy of recent retrieve-and-read models, whilst being significantly faster. |
| Outcome: | The proposed model outperforms baseline models by 5% but trails RePAQ by 15% . it can be configured for size (under 500MB) or speed (over 1K questions per second) while retaining high accuracy. |
Similar Papers
PrimeQA: The Prime Repository for State-of-the-Art Multilingual Question Answering Research and Development (2023.acl-demo)
Copied to clipboard
Avi Sil, Jaydeep Sen, Bhavani Iyer, Martin Franz, Kshitij Fadnis, Mihaela Bornea, Sara Rosenthal, Scott McCarley, Rong Zhang, Vishwajeet Kumar, Yulong Li, Md Arafat Sultan, Riyaz Bhat, Juergen Bross, Radu Florian, Salim Roukos
| Challenge: | Question Answering (QA) is a major area of research in Natural Language Processing (NLP) |
| Approach: | They propose a one-stop and open-source QA repository for question answering . it supports core QA functionalities like retrieval and reading comprehension . they say it will facilitate easy replication of state-of-the-art (SOTA) QA methods . |
| Outcome: | The proposed framework enables easy replication of state-of-the-art (SOTA) QA methods. |
Augmenting Pre-trained Language Models with QA-Memory for Open-Domain Question Answering (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing methods for open-domain question-answering use an open book approach . a recent alternative is to retrieve from a collection of previously-generated question-annwer pairs . |
| Approach: | They propose a new QA system that augments a text-to-text model with a large memory of question-answer pairs and a task for the latent step of question retrieval. |
| Outcome: | The proposed system outperforms closed-book QA and can answer multi-hop questions. |
DoQA - Accessing Domain-Specific FAQs via Conversational QA (2020.acl-main)
Copied to clipboard
| Challenge: | a dataset of 2,437 dialogues and 10,917 QA pairs is used to access domain-specific FAQ information. |
| Approach: | They present a dataset with 2,437 dialogues and 10,917 QA pairs for FAQs . they use the Wizard of Oz method with crowdsourcing to create dialogues using the original post and the original reply. |
| Outcome: | The proposed system can access domain-specific FAQ information without training data. |
Open Domain Question Answering over Tables via Dense Retrieval (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent advances in open-domain QA focus on retrieving textual passages . a retriever designed to handle tabular context can improve retrieval quality . |
| Approach: | They propose a tabular-based retrieval model that improves retrieval quality over a BERT-based retriever. |
| Outcome: | The proposed retriever improves retrieval quality with mined hard negatives over a BERT-based retriever. |
Harvesting and Refining Question-Answer Pairs for Unsupervised QA (2020.acl-main)
Copied to clipboard
| Challenge: | Recent research attempts to extend unsupervised question answering to settings with few or no labeled data available. |
| Approach: | They propose two approaches to improve unsupervised question answering . first, they harvest lexically and syntactically divergent Wikipedia questions to automatically construct a corpus of question-answer pairs . second, they take advantage of the QA model to extract more appropriate answers . |
| Outcome: | The proposed approach outperforms previous unsupervised approaches by a large margin and is competitive with early supervised models. |
Can Generative Pre-trained Language Models Serve As Knowledge Bases for Closed-book QA? (2021.acl-long)
Copied to clipboard
| Challenge: | Existing work is limited in using small benchmarks with high test-train overlaps. |
| Approach: | They construct a dataset of closed-book QA using SQuAD and investigate the performance of BART. |
| Outcome: | Experiments show that pre-trained language models can achieve high performance on closed-book QA tasks. |
NeuralQA: A Usable Library for Question Answering (Contextual Query Expansion + BERT) on Large Datasets (2020.emnlp-demos)
Copied to clipboard
| Challenge: | Existing tools for Question Answering (QA) have challenges that limit their use in practice. |
| Approach: | They propose a library that integrates with existing infrastructure and offers helpful defaults for QA subtasks. |
| Outcome: | NeuralQA integrates well with existing infrastructure and offers helpful defaults for QA subtasks. |
Designing a Minimal Retrieve-and-Read System for Open-Domain Question Answering (2021.naacl-main)
Copied to clipboard
| Challenge: | Open-domain question answering is a task of finding answers to generic factoid questions. |
| Approach: | They propose to use a retrieve-and-read mechanism to drastically reduce the footprint of an open-domain QA system by up to 160x. |
| Outcome: | The proposed method achieves better accuracy than a parametric model with comparable docker-level system size. |
IfQA: A Dataset for Open-domain Question Answering under Counterfactual Presuppositions (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing open-domain QA tasks focus on questions whose answer can be deduced directly from global factual knowledge. |
| Approach: | They propose a dataset where each question is based on a counterfactual presupposition via an "if" clause. |
| Outcome: | The IfQA dataset contains 3,800 questions that were annotated by crowdworkers on relevant Wikipedia passages. |
Relevance-guided Supervision for OpenQA with ColBERT (2021.tacl-1)
Copied to clipboard
| Challenge: | Recent work has focused on learning to retrieve passages for open-domain question answering . if notions of relevance are not tailored to questions, the MRC model will not reliably see the best passages . |
| Approach: | They propose a retrieval model that uses coarse-grained vector representations of questions and passages to adapt it to OpenQA. |
| Outcome: | The proposed system improves OpenQA retrieval on Natural Questions, SQuAD, and TriviaQA. |