CliCR: a Dataset of Clinical Case Reports for Machine Reading Comprehension (N18-1)
Copied to clipboard
| Challenge: | Currently, machine comprehension datasets are extremely scarce for specialized domains. |
| Approach: | They propose a dataset for machine comprehension in the medical domain using clinical case reports with around 100,000 gap-filling queries about these cases. |
| Outcome: | The proposed dataset uses clinical case reports with around 100,000 gap-filling queries about these cases. |
Similar Papers
Clinical Reading Comprehension: A Thorough Analysis of the emrQA Dataset (2020.acl-main)
Copied to clipboard
| Challenge: | Medical professionals often query over clinical notes to find information that can support their decision making. |
| Approach: | They propose to use expert-annotated question templates and existing i2b2 annotations to create emrQA, the first large-scale dataset for question answering based on clinical notes. |
| Outcome: | The proposed system can answer clinical questions without using domain knowledge. |
BioRead: A New Dataset for Biomedical Reading Comprehension (L18-1)
Copied to clipboard
| Challenge: | BioRead is a publicly available cloze-style biomedical machine reading comprehension (MRC) dataset with 16.4 million passage-question instances. |
| Approach: | They propose to build a cloze-style biomedical machine reading comprehension (MRC) dataset with 16.4 million passage-question instances. |
| Outcome: | The proposed method outperforms baselines on bioReadLite and bioASQ, and is currently the best on BioReadLite. |
Dataset for the First Evaluation on Chinese Machine Reading Comprehension (L18-1)
Copied to clipboard
| Challenge: | Existing reading comprehension datasets are mostly in English . |
| Approach: | They propose a Chinese reading comprehension dataset to add diversity to existing reading comprehension data . proposed dataset contains cloze-style reading comprehension and user query reading comprehension . |
| Outcome: | The proposed dataset is based on a Chinese reading comprehension dataset . it includes two types of cloze-style and user query reading comprehension . the proposed dataset hosted the 1st Evaluation on Chinese Machine Reading Comprehension (CMRC-2017) |
M-QALM: A Benchmark to Assess Clinical Reading Comprehension and Knowledge Recall in Large Language Models via Question Answering (2024.findings-acl)
Copied to clipboard
Anand Subramanian, Viktor Schlegel, Abhinav Ramesh Kashyap, Thanh-Tung Nguyen, Vijay Prakash Dwivedi, Stefan Winkler
| Challenge: | Existing studies on adapting large language models to perform a variety of tasks in high-stakes domains such as healthcare lack understanding of the extent and contributing factors that allow them to recall relevant knowledge and combine it with presented information. |
| Approach: | They propose to use multiple choice and abstractive question answering to investigate the extent and contributing factors that allow LLMs to recall relevant knowledge and combine it with presented information in the clinical and biomedical domain. |
| Outcome: | The proposed models perform better on 22 datasets in three generalist and three specialist biomedical sub-domains, and show that they can generalise to unseen sub- domains. |
Machine Reading Comprehension using Case-based Reasoning (2023.findings-emnlp)
Copied to clipboard
Dung Thai, Dhruv Agarwal, Mudit Chaudhary, Wenlong Zhao, Rajarshi Das, Jay-Yoon Lee, Hannaneh Hajishirzi, Manzil Zaheer, Andrew McCallum
| Challenge: | Current state-of-the-art machine readers do not support case-based reasoning . |
| Approach: | They propose a method that extracts a set of similar cases from a nonparametric memory and then predicts an answer by selecting the span in the test context that is most similar to the contextualized representations of answers. |
| Outcome: | The proposed method outperforms baselines on NaturalQuestions and NewsQA by 11.5 and 8.4 EM. |
English Machine Reading Comprehension Datasets: A Survey (2021.emnlp-main)
Copied to clipboard
| Challenge: | a survey of English Machine Reading Comprehension datasets is carried out . the aim is to provide a concise yet informative overview of the landscape . |
| Approach: | They survey 60 English Machine Reading Comprehension datasets to provide a resource for other researchers interested in this problem. |
| Outcome: | The proposed survey covers 60 English MRC datasets with a view to providing a resource for other researchers interested in the problem. |
Towards Medical Machine Reading Comprehension with Structural Knowledge and Plain Text (2020.emnlp-main)
Copied to clipboard
| Challenge: | MRC has achieved significant progress on the open domain in recent years due to large-scale pre-trained language models. |
| Approach: | They propose a machine reading comprehension model which exploits structural medical knowledge and reference medical plain text to improve the exam's accuracy. |
| Outcome: | The proposed model outperforms existing models with a large margin and passes the exam with 61.8% accuracy rate on the test set. |
Comprehensive Multi-Dataset Evaluation of Reading Comprehension (D19-58)
Copied to clipboard
| Challenge: | Recent research aims to facilitate training and evaluation on several reading comprehension datasets at the same time. |
| Approach: | They propose an evaluation server that reports performance on seven diverse reading comprehension datasets and includes synthetic augmentations to test models' ability to handle out-of-domain questions. |
| Outcome: | The evaluation server performs on seven reading comprehension datasets, and collects and includes synthetic augmentations for these datasets to test models' ability to handle out-of-domain questions. |
ESTER: A Machine Reading Comprehension Dataset for Reasoning about Event Semantic Relations (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent event-centric reading comprehension datasets focus mostly on event arguments or temporal relations. |
| Approach: | They propose a machine reading comprehension dataset that leverages natural language queries to reason about the five most common event semantic relations. |
| Outcome: | The proposed dataset shows that current SOTA systems achieve 22.1%, 63.3% and 83.5% for token-based exact-match, **F1** and event-based **HIT@1** scores. |
A Sentence Cloze Dataset for Chinese Machine Reading Comprehension (2020.coling-main)
Copied to clipboard
| Challenge: | Using cloze-style reading comprehension, Chinese machine reading comprehension datasets are becoming more and more popular . a new task is proposed to fill the right candidate sentence into the passage with several blanks . |
| Approach: | They propose a Chinese task to fill the right candidate sentence into a passage with blanks . they build a dataset to evaluate the difficulty of the task and make fake candidates . |
| Outcome: | The proposed task fills the right candidate sentence into the passage with blanks . the proposed dataset contains over 100K blanks within over 10K passages based on Chinese narrative stories . |