| Challenge: | Recent proposed systems for open-domain question answering (OpenQA) require large amounts of training data to achieve state-of-the-art performance. |
| Approach: | They propose an efficient annotation strategy that increases passage retrieval accuracy@10 by 10.55 p.p. while reducing the annotation cost by 82%. |
| Outcome: | The proposed approach increases passage retrieval accuracy @10 by 10.55 p.p. while reducing the annotation cost by 82%. |
Similar Papers
XQA: A Cross-lingual Open-domain Question Answering Dataset (P19-1)
Copied to clipboard
| Challenge: | Open-domain question answering aims to answer questions through text retrieval and reading comprehension . but, the success of these models relies on a massive volume of training data, which is not available in other languages . a new dataset aims at investigating cross-lingual OpenQA . |
| Approach: | They propose to use a dataset for cross-lingual OpenQA research to test models . they use XQA dataset to train models with large volumes of labeled data . |
| Outcome: | The proposed model achieves best results in almost all target languages while the performance is lower than that of English. |
Developing PUGG for Polish: A Modern Approach to KBQA, MRC, and IR Dataset Construction (2024.findings-acl)
Copied to clipboard
Albert Sawczyn, Katsiaryna Viarenich, Konrad Wojtasik, Aleksandra Domogała, Marcin Oleksy, Maciej Piasecki, Tomasz Kajdanowicz
| Challenge: | Existing KBQA datasets are outdated and inefficient in human labor, and assisting tools like Large Language Models (LLM) are not utilized to reduce the workload. |
| Approach: | They propose a semi-automated question answering task that uses structured knowledge graphs to answer extensive knowledge-intensive questions. |
| Outcome: | The proposed approach includes KBQA, MRC, and Information Retrieval tasks for low-resource languages. |
CCQA: A New Web-Scale Question Answering Dataset for Model Pre-Training (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing approaches to answer open domain questions rely on unlabeled text or synthetically generated question-answer pairs. |
| Approach: | They propose a large-scale open-domain question-answering dataset based on the Common Crawl project that can be used to in-domain pre-train popular language models. |
| Outcome: | The proposed dataset achieves promising results in zero-shot, low resource and fine-tuned settings across multiple tasks, models and benchmarks. |
MedQA-SWE - a Clinical Question & Answer Dataset for Swedish (2024.lrec-main)
Copied to clipboard
| Challenge: | MedQA-SWE is a clinical question & answering dataset in Swedish . it was created from exams aimed at evaluating doctors’ clinical understanding and decision making . |
| Approach: | They propose to create a multiple choice, clinical question & answering (Q&A) dataset in Swedish consisting of 3,180 questions. |
| Outcome: | The proposed dataset includes 3,180 questions and is the first open-source clinical Q&A dataset in Swedish. |
UQA: Corpus for Urdu Question Answering (2024.lrec-main)
Copied to clipboard
| Challenge: | Urdu is a low-resource language with over 70 million native speakers . expanding the reach of NLP to languages other than English is crucial for advancing multilingual AI systems. |
| Approach: | They introduce a novel dataset for question answering and text comprehension in Urdu . they use a technique called EATS which preserves the answer spans in translated context paragraphs . |
| Outcome: | The proposed dataset preserves answer spans in translated context paragraphs. |
MKQA: A Linguistically Diverse Benchmark for Multilingual Open Domain Question Answering (2021.tacl-1)
Copied to clipboard
| Challenge: | Existing multilingual QA datasets lack linguistic diversity and comparable evaluation between languages. |
| Approach: | They propose a multilingual question-answer evaluation set with 10k English queries and human translations of them into 25 additional languages and dialects. |
| Outcome: | The proposed model is based on a multilingual knowledge questions and answers evaluation set with 26 languages. |
KazQAD: Kazakh Open-Domain Question Answering Dataset (2024.lrec-main)
Copied to clipboard
| Challenge: | KazQAD contains just under 6,000 unique questions with extracted short answers and nearly 12,000 passage-level relevance judgements. |
| Approach: | They introduce a Kazakh open-domain question answering dataset that can be used in reading comprehension and full ODQA settings. |
| Outcome: | The proposed dataset can be used in reading comprehension and full ODQA settings, as well as for information retrieval experiments. |
NeuralQA: A Usable Library for Question Answering (Contextual Query Expansion + BERT) on Large Datasets (2020.emnlp-demos)
Copied to clipboard
| Challenge: | Existing tools for Question Answering (QA) have challenges that limit their use in practice. |
| Approach: | They propose a library that integrates with existing infrastructure and offers helpful defaults for QA subtasks. |
| Outcome: | NeuralQA integrates well with existing infrastructure and offers helpful defaults for QA subtasks. |
Open-Domain Question Answering (2020.acl-tutorials)
Copied to clipboard
| Challenge: | tutorial provides a comprehensive overview of cutting-edge research in open-domain question answering (QA) |
| Approach: | tutorial provides a comprehensive overview of cutting-edge research in open-domain question answering . focus will shift to cutting- edge models proposed for open- domain QA . |
| Outcome: | The tutorial will cover cutting-edge research in open-domain question answering (QA) it will cover two-stage retriever-reader approaches, dense retriever and end-to-end training, and retriever free methods . |
PeerQA: A Scientific Question Answering Dataset from Peer Reviews (2025.naacl-long)
Copied to clipboard
| Challenge: | a dataset of 579 QA pairs from 208 scientific articles contains answers that reviewers raised while thoroughly examining the scientific article. |
| Approach: | They propose a dataset that contains questions that reviewers raised while thoroughly examining the scientific article. |
| Outcome: | The proposed dataset contains 579 QA pairs from 208 academic articles . the results show that decontextualization approaches improve retrieval performance . |