Dirk Weissenborn, Pasquale Minervini, Isabelle Augenstein, Johannes Welbl, Tim Rocktäschel, Matko Bošnjak, Jeff Mitchell, Thomas Demeester, Tim Dettmers, Pontus Stenetorp, Sebastian Riedel
| Challenge: | Many Machine Reading and Natural Language Understanding tasks require reading supporting text in order to answer questions. |
| Approach: | They propose a framework for Machine Reading that allows for quick prototyping by component reuse and evaluation of new models on existing datasets. |
| Outcome: | The proposed framework supports question answering, natural language inference and link prediction tasks. |
Similar Papers
Proceedings of the 2nd Workshop on Machine Reading for Question Answering (D19-58)
Copied to clipboard
| Challenge: | a workshop focuses on machine reading for question answering . despite recent progress, there is much to be desired about these datasets and systems . |
| Approach: | This year, they present a shared task on machine reading for question answering . they adapt and unified 18 distinct question answering datasets into the same format . |
| Outcome: | The proposed system achieves an average F1 score of 72.5 on the held-out datasets. |
SQL Generation via Machine Reading Comprehension (2020.coling-main)
Copied to clipboard
| Challenge: | Text-to-SQL systems can generate SQL queries given natural language questions. |
| Approach: | They propose a method that formulates a question answering problem as a query answering problem where different slots are predicted by a unified machine reading comprehension (MRC) model. |
| Outcome: | The proposed method can achieve competitive results on WikiSQL, suggesting it being a promising direction for text-to-SQl. |
A Multi-answer Multi-task Framework for Real-world Machine Reading Comprehension (D18-1)
Copied to clipboard
| Challenge: | Existing models of machine reading comprehension (MRC) are based on cloze style questions or crowdworkers given a short passage from well-edited sources. |
| Approach: | They propose a multi-answer multi-task framework that uses multiple reference answers for multiple questions. |
| Outcome: | The proposed model increases the ROUGE-L score on the DuReader dataset from 44.18, the previous state-of-the-art, to 51.09 . |
D-NET: A Pre-Training and Fine-Tuning Framework for Improving the Generalization of Machine Reading Comprehension (D19-58)
Copied to clipboard
Hongyu Li, Xiyuan Zhang, Yibing Liu, Yiming Zhang, Quan Wang, Xiangyang Zhou, Jing Liu, Hua Wu, Haifeng Wang
| Challenge: | MRC requires machines to understand text and answer questions about the text. |
| Approach: | They propose a simple system Baidu submitted for MRQA 2019 Shared Task that focused on generalization of machine reading comprehension (MRC) models. |
| Outcome: | The proposed system is ranked at top 1 of all participants in terms of averaged F1 score. |
Improving Machine Reading Comprehension with General Reading Strategies (N19-1)
Copied to clipboard
| Challenge: | Recent studies have shown that reading strategies improve comprehension levels for readers lacking adequate prior knowledge. |
| Approach: | They propose three general strategies to improve machine reading comprehension (MRC) by fine-tuning a pre-trained model with strategies and a target task. |
| Outcome: | The proposed models improve non-extractive machine reading comprehension (MRC) on the largest general domain multiple-choice dataset RACE. |
Proto-lm: A Prototypical Network-Based Framework for Built-in Interpretability in Large Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for interpreting LLMs are post hoc and focus on low-level features and lack of explainability at higher-level text units. |
| Approach: | They propose a prototypical network-based white-box framework that allows LLMs to learn immediately interpretable embeddings during the fine-tuning stage while maintaining competitive performance. |
| Outcome: | The proposed framework can learn interpretable embeddings during the fine-tuning stage while maintaining competitive performance. |
Document Modeling with Graph Attention Networks for Multi-grained Machine Reading Comprehension (2020.acl-main)
Copied to clipboard
| Challenge: | Existing approaches to machine reading comprehension treat documents at their hierarchical nature, ignoring their dependencies. |
| Approach: | They propose a machine reading comprehension benchmark with two-grained answers . they use graph attention networks to model documents at their hierarchical nature . |
| Outcome: | The proposed framework outperforms existing systems at long and short answer criteria. |
ReTraCk: A Flexible and Efficient Framework for Knowledge Base Question Answering (2021.acl-demo)
Copied to clipboard
| Challenge: | Existing neural semantic parsing methods for knowledge base question answering are lacking . a generic and extensible framework is lacking for KBQA. |
| Approach: | They propose a neural semantic parsing framework for large scale knowledge base question answering . they propose 'retriever-transducer-checker' framework that provides a retriever and a transducer . |
| Outcome: | The proposed framework is ranked at top1 overall performance on the GrailQA leaderboard and achieves competitive performance on typical WebQuestionsSP benchmark. |
SkillQG: Learning to Generate Question for Reading Comprehension Assessment (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing question generation systems focus on the literal nature of questions and rarely consider comprehension types of the generated questions. |
| Approach: | They propose a question generation framework with controllable comprehension types for machine reading comprehension models. |
| Outcome: | Empirical results show that SkillQG outperforms baselines in quality, relevance, and skill-controllability while showing a performance boost in downstream question answering task. |
Machine Reading Comprehension using Case-based Reasoning (2023.findings-emnlp)
Copied to clipboard
Dung Thai, Dhruv Agarwal, Mudit Chaudhary, Wenlong Zhao, Rajarshi Das, Jay-Yoon Lee, Hannaneh Hajishirzi, Manzil Zaheer, Andrew McCallum
| Challenge: | Current state-of-the-art machine readers do not support case-based reasoning . |
| Approach: | They propose a method that extracts a set of similar cases from a nonparametric memory and then predicts an answer by selecting the span in the test context that is most similar to the contextualized representations of answers. |
| Outcome: | The proposed method outperforms baselines on NaturalQuestions and NewsQA by 11.5 and 8.4 EM. |