Wenhan Xiong, Jiawei Wu, Hong Wang, Vivek Kulkarni, Mo Yu, Shiyu Chang, Xiaoxiao Guo, William Yang Wang
| Challenge: | Social media is becoming an important realtime information source, especially during natural disasters and emergencies. |
| Approach: | They present a large-scale dataset for question answering over social media data . they gather tweets used by journalists and ask human annotators to write questions upon them . |
| Outcome: | The proposed dataset shows that neural models that perform well on formal texts are limited in their performance . the proposed model is still lagging behind human performance with a large margin . |
Similar Papers
PragmatiCQA: A Dataset for Pragmatic Question Answering in Conversations (2023.findings-acl)
Copied to clipboard
| Challenge: | Mars? - PragmatiCQA |
| Approach: | Mars? - The Paper . |
| Outcome: | The proposed dataset features 6873 QA pairs that explores pragmatic reasoning in conversations over a diverse set of topics. |
ComQA: A Community-sourced Dataset for Complex Factoid Question Answering with Paraphrase Clusters (N19-1)
Copied to clipboard
| Challenge: | ComQA dataset captures question phenomena and the diverse ways in which they are formulated. |
| Approach: | They propose a large dataset of real user questions that captures question phenomena and the diverse ways in which they are formulated. |
| Outcome: | The proposed dataset can be a driver of future research on factoid question answering (QA). |
NeuralQA: A Usable Library for Question Answering (Contextual Query Expansion + BERT) on Large Datasets (2020.emnlp-demos)
Copied to clipboard
| Challenge: | Existing tools for Question Answering (QA) have challenges that limit their use in practice. |
| Approach: | They propose a library that integrates with existing infrastructure and offers helpful defaults for QA subtasks. |
| Outcome: | NeuralQA integrates well with existing infrastructure and offers helpful defaults for QA subtasks. |
ForecastQA: A Question Answering Challenge for Event Forecasting with Temporal Text Data (2021.acl-long)
Copied to clipboard
| Challenge: | Existing automated forecasting studies rely on structured data to predict future events. |
| Approach: | They propose a question-answering task that limits access to unstructured text data . they use a crowdsourced dataset to form a restricted-domain, multiple-choice, question-announcement task . |
| Outcome: | The proposed model achieves 61.0% accuracy on the dataset, which still lags behind human performance by about 19%. |
TIGQA: An Expert-Annotated Question-Answering Dataset in Tigrinya (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing annotated datasets for NLP tasks in languages with limited resources are limited. |
| Approach: | They propose to use machine translation to convert existing Tigrinya dataset into a Tigrina dataset in SQuAD format. |
| Outcome: | The proposed dataset is an expert-annotated Tigrinya dataset with 2,685 question-answer pairs covering 122 diverse topics. |
FocusQA: Open-Domain Question Answering with a Context in Focus (2022.findings-emnlp)
Copied to clipboard
Gianni Barlacchi, Ivano Lauriola, Alessandro Moschitti, Marco Del Tredici, Xiaoyu Shen, Thuy Vu, Bill Byrne, Adrià de Gispert
| Challenge: | a new method for question answering with a context in focus simulates a free interaction with QA systems. |
| Approach: | They introduce question answering with a cotext in focus task that simulates a free interaction with QA systems. |
| Outcome: | The proposed model outperforms state-of-the-art models for question answering with a context in focus up to 21.3% absolute points. |
Multi-Relational Question Answering from Narratives: Machine Reading and Reasoning in Simulated Worlds (P18-1)
Copied to clipboard
| Challenge: | Question Answering (QA) has primarily focused on knowledge bases or free text as a source of knowledge. |
| Approach: | They propose a task of multi-relational QA over personal narrative using text worlds . they generate and release a lightweight Python-based framework for easily generating additional worlds and narrative . |
| Outcome: | The proposed framework combines elements of structured QA over knowledge bases and unstructured QA . it generates and analyzes five diverse datasets with dynamic narrative . the framework is lightweight and easy to use . |
TWEETSUM: Event oriented Social Summarization Dataset (2020.coling-main)
Copied to clipboard
| Challenge: | Developing social summarization systems is becoming more and more critical . but, the publicly available and high-quality large scale social summaries are rare . |
| Approach: | They propose to build a social summarization dataset using twitter's hot events . they collect user relations, hashtags and user profiles to evaluate their summarizing methods . |
| Outcome: | The proposed dataset is based on a dataset from twitter with 12 real world hot events with 44,034 tweets and 11,240 users. |
NoiseQA: Challenge Set Evaluation for User-Centric Question Answering (2021.eacl-main)
Copied to clipboard
| Challenge: | Question-Answering (QA) systems are deployed in the real world . a lack of research attention has been devoted to studying the issues that arise when people use QA systems. |
| Approach: | They show that component components that precede an answering engine can introduce varied and considerable sources of error. |
| Outcome: | The proposed evaluations highlight the need for QA evaluation to expand to consider real-world use. |
QAConv: Question Answering on Informative Conversations (2022.acl-long)
Copied to clipboard
| Challenge: | Experimental results show that state-of-the-art pretrained QA systems have limited zero-shot performance and tend to predict our questions as unanswerable. |
| Approach: | They propose a question-answering dataset that uses conversations as a knowledge source. |
| Outcome: | The proposed dataset provides a training and evaluation testbed to facilitate QA on conversations research. |