| Challenge: | Developing such datasets is important for the development and evaluation of Icelandic QA systems. |
| Approach: | They present the first extractive question answering dataset for Icelandic, Natural Questions in Icelandic. |
| Outcome: | The proposed dataset is a valuable resource for Icelandic which is being evaluated by a team of researchers. |
Similar Papers
NativQA: Multilingual Culturally-Aligned Natural Query for LLMs (2025.findings-acl)
Copied to clipboard
Md. Arid Hasan, Maram Hasanain, Fatema Ahmad, Sahinur Rahman Laskar, Sunaya Upadhyay, Vrunda N Sukhadia, Mucahid Kutlu, Shammur Absar Chowdhury, Firoj Alam
| Challenge: | Existing frameworks for QA datasets lack regional specificity and cultural specificity. |
| Approach: | They propose a framework to quench native language QA datasets in native languages for LLM evaluation and tuning. |
| Outcome: | The proposed framework is scalable, language-independent and can be used to build culturally and regionally aligned QA datasets in native languages. |
A guide to the dataset explosion in QA, NLI, and commonsense reasoning (2020.coling-tutorials)
Copied to clipboard
| Challenge: | a tutorial aims to provide an up-to-date guide to the recent datasets . the target audience is the NLP practitioners who are lost in dozens of the recent data sets. |
| Approach: | This tutorial provides an up-to-date guide to the recent datasets . it surveys old and new methodological issues with dataset construction . |
| Outcome: | This tutorial aims to provide an up-to-date guide to the recent datasets . it surveys the old and new methodological issues with dataset construction . |
BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions (N19-1)
Copied to clipboard
| Challenge: | In this paper we build a reading comprehension dataset of yes/no questions that are naturally occurring . they often query for complex, non-factoid information, and require difficult entailment-like inference to solve. |
| Approach: | They build a reading comprehension dataset of yes/no questions that are naturally occurring . they find they are unexpectedly challenging and require difficult inferences to solve . |
| Outcome: | The proposed method achieves 80.4% accuracy compared to 90% accuracy of human annotators and 62% majority-baseline. |
From Multiple-Choice to Extractive QA: A Case Study for English and Arabic (2025.coling-main)
Copied to clipboard
Teresa Lynn, Malik H. Altakrori, Samar M. Magdy, Rocktim Jyoti Das, Chenyang Lyu, Mohamed Nasr, Younes Samih, Kirill Chirkunov, Alham Fikri Aji, Preslav Nakov, Shantanu Godbole, Salim Roukos, Radu Florian, Nizar Habash
| Challenge: | Recent years have brought about very fast developments in Natural Language Processing (NLP), but many other languages are overlooked due to limited resources. |
| Approach: | They propose to repurpose a multilingual BELEBELE dataset for a task of extractive QA in the style of machine reading comprehension. |
| Outcome: | The proposed approach could be used to extract QA in the style of machine reading comprehension. |
Towards more equitable question answering systems: How much more data do you need? (2021.acl-short)
Copied to clipboard
| Challenge: | Question answering datasets in English are relatively new, but lack of linguistic diversity in the field is a challenge. |
| Approach: | They propose to use translation and cross-lingual transfer to produce QA systems in multiple languages to improve their performance. |
| Outcome: | The proposed approaches take advantage of existing resources to produce QA systems in multiple languages. |
Unsupervised Question Answering by Cloze Translation (P19-1)
Copied to clipboard
| Challenge: | Existing QA datasets only available for limited domains and languages. |
| Approach: | They propose to generate context, question and answer triples in an unsupervised manner and synthesize extractive QA training data automatically. |
| Outcome: | The proposed approach outperforms existing QA models on a common EQA benchmark dataset. |
DBQR-QA: A Question Answering Dataset on a Hybrid of Database Querying and Reasoning (2024.findings-acl)
Copied to clipboard
| Challenge: | Question answering (QA) is a fundamental task in the field of Natural Language Processing (NLP). |
| Approach: | They propose a database querying and reasoning dataset for question answering that is designed to accommodate sequential questions and multi-hop queries. |
| Outcome: | The proposed dataset better mirrors the dynamics of real-world information retrieval and analysis with a particular focus on the financial reports of US companies. |
Can LLM Generate Culturally Relevant Commonsense QA Data? Case Study in Indonesian and Sundanese (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly being used to generate synthetic data for training and evaluating models. |
| Approach: | They investigate the effectiveness of using Large Language Models to generate culturally relevant commonsense QA datasets for Indonesian and Sundanese languages using both LLMs and human annotators. |
| Outcome: | The proposed model generates 4.5K questions per language, compared with 4.5k for Indonesian and 4.5km for Sundanese. |
QA-NatVer: Question Answering for Natural Logic-based Fact Verification (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent work has focused on natural logic, which operates directly on natural language by capturing the semantic relation of spans between an aligned claim and its evidence via set-theoretic operators. |
| Approach: | They propose to use question answering to predict natural logic operators using generalization capabilities of instruction-tuned language models. |
| Outcome: | The proposed approach outperforms the best baseline on a Danish verification dataset by 4.3 accuracy points. |
GameQA: Gamified Mobile App Platform for Building Multiple-Domain Question-Answering Datasets (2023.eacl-demo)
Copied to clipboard
Njall Skarphedinsson, Breki Gudmundsson, Steinar Smari, Marta Kristin Larusdottir, Hafsteinn Einarsson, Abuzar Khan, Eric Nyberg, Hrafn Loftsson
| Challenge: | a common problem with question-answering datasets is that they require annotators to source answers from the internet . a crowd-sourcing platform is available for low-resource languages, but it is limited in terms of information available. |
| Approach: | They propose a crowd-sourcing platform to gather multiple-domain QA data for low-resource languages. |
| Outcome: | The proposed platform rivals large QA datasets for high-resource languages in size and answerability. |