| Challenge: | Several recent MRC models employ multi-step reasoning . we show that the use of a stochastic prediction dropout improves robustness . |
| Approach: | They propose a stochastic answer network that simulates multi-step reasoning in machine reading comprehension. |
| Outcome: | The proposed model improves robustness and results competitive with state-of-the-art models on the Stanford Question Answering Dataset and Microsoft MAchine Reading COmprehension Dataset. |
Similar Papers
Multi-Passage Machine Reading Comprehension with Cross-Passage Answer Verification (P18-1)
Copied to clipboard
| Challenge: | Recent years have seen rapid growth in the MRC community . MRC is believed to be a crucial step in building a general intelligent agent . |
| Approach: | They propose an end-to-end neural model that enables multiple passages to verify each other based on their content representations. |
| Outcome: | The proposed model outperforms the baseline on the English MS-MARCO dataset and the Chinese DuReader dataset, and achieves state-of-the-art performance on both datasets. |
A Multi-Type Multi-Span Network for Reading Comprehension that Requires Discrete Reasoning (D19-1)
Copied to clipboard
| Challenge: | Existing models for reading comprehension and question answering do not support discrete reasoning abilities. |
| Approach: | They propose a reading comprehension model that uses a multi-type answer predictor and a multiple-span extraction method to produce one or multiple text spans. |
| Outcome: | The proposed model achieves 79.9 F1 on the DROP hidden test set, creating new state-of-the-art results. |
Cut to the Chase: A Context Zoom-in Network for Reading Comprehension (D18-1)
Copied to clipboard
| Challenge: | Recent deep-learning based models suffer from reasoning over long documents and do not trivially generalize to cases where the answer is not present as a span. |
| Approach: | They propose a novel context zoom-in network (ConZNet) that can skip through irrelevant parts of a document and generate an answer using only the relevant regions of text. |
| Outcome: | The proposed architecture outperforms state-of-the-art results by 12.62% (ROUGE-L) relative improvement on the recently proposed and challenging RC dataset ‘NarrativeQA’. |
Machine Reading Comprehension using Case-based Reasoning (2023.findings-emnlp)
Copied to clipboard
Dung Thai, Dhruv Agarwal, Mudit Chaudhary, Wenlong Zhao, Rajarshi Das, Jay-Yoon Lee, Hannaneh Hajishirzi, Manzil Zaheer, Andrew McCallum
| Challenge: | Current state-of-the-art machine readers do not support case-based reasoning . |
| Approach: | They propose a method that extracts a set of similar cases from a nonparametric memory and then predicts an answer by selecting the span in the test context that is most similar to the contextualized representations of answers. |
| Outcome: | The proposed method outperforms baselines on NaturalQuestions and NewsQA by 11.5 and 8.4 EM. |
GCRC: A New Challenging MRC Dataset from Gaokao Chinese for Explainable Evaluation (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing machine reading comprehension datasets lack an explainable evaluation of systems' reasoning capabilities. |
| Approach: | They propose a dataset with multi-choice questions that evaluates MRC systems' reasoning process . they use sentence-level relevant supporting facts, error reason of distractors to evaluate MRC . |
| Outcome: | The proposed dataset is more challenging and useful for identifying limitations of existing MRC systems in an explainable way. |
ARES: A Reading Comprehension Ensembling Service (2020.emnlp-demos)
Copied to clipboard
Anthony Ferritto, Lin Pan, Rishav Chakravarti, Salim Roukos, Radu Florian, J. William Murdock, Avi Sil
| Challenge: | ARES is a machine reading comprehension (MRC) demonstration system which utilizes an ensemble of models to increase F1 by 2.3 points. |
| Approach: | They propose a machine reading comprehension (MRC) demonstration system which utilizes an ensemble of models to increase F1 by 2.3 points. |
| Outcome: | The proposed system increases F1 by 2.3 points on a short answer task using an ensemble of models. |
Answer Uncertainty and Unanswerability in Multiple-Choice Machine Reading Comprehension (2022.findings-acl)
Copied to clipboard
| Challenge: | Machine reading comprehension (MRC) systems focus on selecting the correct answer to a question given a context paragraph. |
| Approach: | They propose to use machine reading comprehension (MRC) to assess the ability of systems to understand natural language. |
| Outcome: | The proposed system outperforms a system built with an NOA option . the results show that the system is not confident about the NOA choice . |
A Multi-answer Multi-task Framework for Real-world Machine Reading Comprehension (D18-1)
Copied to clipboard
| Challenge: | Existing models of machine reading comprehension (MRC) are based on cloze style questions or crowdworkers given a short passage from well-edited sources. |
| Approach: | They propose a multi-answer multi-task framework that uses multiple reference answers for multiple questions. |
| Outcome: | The proposed model increases the ROUGE-L score on the DuReader dataset from 44.18, the previous state-of-the-art, to 51.09 . |
AdaLoGN: Adaptive Logic Graph Network for Reasoning-Based Machine Reading Comprehension (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods and limitations for machine reading comprehension are insufficient for logical reasoning over text. |
| Approach: | They propose a neural-symbolic approach which passes messages over a graph representing logical relations between text units to predict an answer. |
| Outcome: | The proposed approach outperforms existing methods on ReClor and LogiQA. |
Joint Training of Candidate Extraction and Answer Selection for Reading Comprehension (P18-1)
Copied to clipboard
| Challenge: | Various advanced neural models have been proposed for reading comprehension, but most models ignore its relations with other answer candidates. |
| Approach: | They propose to model reading comprehension as an extract-then-select two-stage procedure . they first extract answer candidates from passages, then select the final answer by combining information from all candidates. |
| Outcome: | The proposed approach improves state-of-the-art performance on open-domain reading comprehension datasets. |