Teaching Machine Comprehension with Compositional Explanations (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in machine reading comprehension rely heavily on large-scale annotated corpora, which are timeconsuming and costly to collect. |
| Approach: | They propose to use semi-structured explanations to “teach” machines reading comprehension using a small number of semi-structural explanations that explicitly inform machines why answer spans are correct. |
| Outcome: | The proposed method achieves 70.14% F1 score with supervision from 26 explanations on the SQuAD dataset, comparable to plain supervised learning using 1,100 labeled instances yielding a 12x speed up. |
Similar Papers
Benchmarking Machine Reading Comprehension: A Psychological Perspective (2021.eacl-main)
Copied to clipboard
| Challenge: | MRC is a task that tests the ability of a machine to read and understand unstructured text. |
| Approach: | They propose a theoretical basis for the design of MRC datasets based on psychology and psychometrics and propose shortcut-proof questions and explanations as a part of the task design. |
| Outcome: | The proposed datasets should evaluate the model's ability to understand context-dependent situations and ensure substantive validity by shortcut-proof questions and explanation as a part of the task design. |
Coreference Reasoning in Machine Reading Comprehension (2021.acl-long)
Copied to clipboard
| Challenge: | Existing datasets for machine reading comprehension do not reflect the natural distribution and, consequently, the challenges of coreference reasoning. |
| Approach: | They propose to use existing coreference resolution datasets to train machine reading comprehension models to better reflect the natural distribution and, consequently, the challenges of coreference reasoning. |
| Outcome: | The proposed method improves the performance of state-of-the-art models on a set of coreference-related datasets. |
A Framework for Evaluation of Machine Reading Comprehension Gold Standards (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing literature on machine reading comprehension (MRC) data is limited on the data design of gold standards. |
| Approach: | They propose a framework to investigate linguistic features, lexical cues and ambiguity in MRC gold standards. |
| Outcome: | The proposed framework investigates the present linguistic features, required reasoning and background knowledge and factual correctness on the one hand, and the presence of lexical cues as a lower bound for the requirement of understanding on the other. |
Self-Teaching Machines to Read and Comprehend with Large-Scale Multi-Subject Question-Answering Data (2021.findings-emnlp)
Copied to clipboard
| Challenge: | despite considerable progress, most machine reading comprehension tasks lack sufficient training data to fully exploit powerful deep neural network models. |
| Approach: | They propose to use QA data to generate more training data for machine reading comprehension tasks by crowdsourcing . they first collect a large-scale multiple-choice QA dataset for Chinese, ExamQA, and then use incomplete, yet relevant snippets returned by a web search engine as the context for each QA instance. |
| Outcome: | The proposed model improves a Chinese MRC task with +5.1% accuracy and +3.8% exact match. |
A Self-Training Method for Machine Reading Comprehension with Soft Evidence Extraction (2020.acl-main)
Copied to clipboard
| Challenge: | Existing models for machine reading comprehension lack evidence labels for training models. |
| Approach: | They propose a method which supervises the evidence extractor with auto-generated evidence labels in an iterative process. |
| Outcome: | The proposed method improves on three MRC tasks on seven datasets. |
Why Machine Reading Comprehension Models Learn Shortcuts? (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing studies show that many MRC models learn shortcuts to outwit benchmarks, but the performance is unsatisfactory in real-world applications. |
| Approach: | They propose to use shortcut questions to analyze learning difficulty of MRC models . they propose to analyze the learning difficulty regarding shortcut and challenging questions . |
| Outcome: | The proposed methods show that a large proportion of shortcut questions in training data make models rely on shortcut tricks excessively. |
Machine Reading Comprehension using Case-based Reasoning (2023.findings-emnlp)
Copied to clipboard
Dung Thai, Dhruv Agarwal, Mudit Chaudhary, Wenlong Zhao, Rajarshi Das, Jay-Yoon Lee, Hannaneh Hajishirzi, Manzil Zaheer, Andrew McCallum
| Challenge: | Current state-of-the-art machine readers do not support case-based reasoning . |
| Approach: | They propose a method that extracts a set of similar cases from a nonparametric memory and then predicts an answer by selecting the span in the test context that is most similar to the contextualized representations of answers. |
| Outcome: | The proposed method outperforms baselines on NaturalQuestions and NewsQA by 11.5 and 8.4 EM. |
Improving Machine Reading Comprehension with Contextualized Commonsense Knowledge (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods to improve machine reading comprehension (MRC) tasks require unstated knowledge to perform well. |
| Approach: | They propose to extract a new kind of structured knowledge from scripts and use it to improve machine reading comprehension (MRC) They propose a teacher-student paradigm to facilitate the transfer of knowledge in weakly-labeled MRC data. |
| Outcome: | The proposed method outperforms methods that use weakly-labeled data and improves a state-of-the-art baseline by 4.3% in accuracy on a Chinese multiple-choice MRC dataset C3. |
A Span-Extraction Dataset for Chinese Machine Reading Comprehension (D19-1)
Copied to clipboard
| Challenge: | Existing reading comprehension datasets are mostly in English . MRC is a new field of research that aims to comprehend the context of articles and answer the questions based on them. |
| Approach: | They propose a Span-Extraction dataset for Chinese machine reading comprehension to add language diversities to existing reading comprehension datasets. |
| Outcome: | The proposed dataset is composed of 20,000 real questions annotated on Wikipedia paragraphs by human experts. |
Incorporating Syntax and Frame Semantics in Neural Network for Machine Reading Comprehension (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods for machine reading comprehension rely on manually defined features and are difficult to generalize to other tasks. |
| Approach: | They propose a Syntax and Frame Semantics model for Machine Reading Comprehension which takes full advantage of syntax and frame semantics to get richer text representation. |
| Outcome: | The proposed model outperforms ten state-of-the-art models on machine reading comprehension tasks. |