WorldTree: A Corpus of Explanation Graphs for Elementary Science Questions supporting Multi-hop Inference (L18-1)
Copied to clipboard
| Challenge: | Existing methods of automated inference do not provide enough gold explanations to train models . standardized science exams are a challenge task for question answering . |
| Approach: | They propose to manually construct a corpus of explanations for standardized science exams . they also provide an explanation-centered tablestore that contains the knowledge to construct these explanations . |
| Outcome: | The proposed model provides detailed explanations for standardized science exams . the authors show that the proposed model can be trained on the basis of gold explanations . |
Similar Papers
WorldTree V2: A Corpus of Science-Domain Structured Explanations and Inference Patterns supporting Multi-Hop Inference (2020.lrec-1)
Copied to clipboard
Zhengnan Xie, Sebastian Thiem, Jaycie Martin, Elizabeth Wainwright, Steven Marmorstein, Peter Jansen
| Challenge: | Standardized science questions require combining an average of 6 facts and as many as 16 facts to answer and explain. |
| Approach: | They propose to combine an average of 6 facts and as many as 16 facts to produce an answer for complex questions. |
| Outcome: | The proposed model is based on a corpus of 5,114 standardized science exam questions . it uses multi-fact explanations that combine science knowledge and world knowledge . |
TextGraphs 2019 Shared Task on Multi-Hop Inference for Explanation Regeneration (D19-53)
Copied to clipboard
| Challenge: | Detailed extended analyses of all submitted systems showed large relative improvements in accessing the most challenging multi-hop inference problems, while absolute performance remains low. |
| Approach: | The Shared Task on Multi-Hop Inference for Explanation Regeneration asks participants to regenerate detailed gold explanations for elementary science questions by selecting facts from a knowledge base of semi-structured tables. |
| Outcome: | The top-performing system achieved a mean average precision of 0.56 . the task combines facts from a knowledge base and supervised training data . |
Explainable Inference Over Grounding-Abstract Chains for Science Questions (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing inference models for science questions are black-box by nature, lacking explanations for their predictions. |
| Approach: | They propose an explainable inference approach for science questions by reasoning on grounding and abstract inference chains. |
| Outcome: | The proposed model generates plausible explanations for science questions using a weighted graph of relevant facts and a Bayesian Optimisation formalism. |
Extracting Common Inference Patterns from Semi-Structured Explanations (D19-60)
Copied to clipboard
| Challenge: | Multi-hop inference suffers from semantic drift, or the tendency for chains of reasoning to "drift"' to unrelated topics. |
| Approach: | They propose to extract large high-confidence multi-hop inference patterns from a corpus of explanations by abstracting large-scale structure from logical sentences. |
| Outcome: | The proposed method extracts large high-confidence multi-hop inference patterns from a “matter” subset of elementary science exam questions. |
Explaining Answers with Entailment Trees (2021.emnlp-main)
Copied to clipboard
Bhavana Dalvi, Peter Jansen, Oyvind Tafjord, Zhengnan Xie, Hannah Smith, Leighanna Pipatanangkura, Peter Clark
| Challenge: | ENTAILMENTBANK is the first dataset to contain multistep entailment trees. |
| Approach: | They propose to generate explanations in the form of entailment trees, a tree of multipremise entanglements steps from facts that are known to the hypothesis of interest. |
| Outcome: | The proposed model can generate explanations in the form of entailment trees . this is a tree of multipremise enttailment steps from facts known to the hypothesis of interest. |
Unification-based Reconstruction of Multi-hop Explanations for Science Questions (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing approaches build explanations considering each question in isolation, but new approach leverages explanatory patterns emerging in scientific explanations. |
| Approach: | They propose a framework for reconstructing multi-hop explanations in science Question Answering . they integrate lexical relevance with the notion of unification power to rank atomic facts . |
| Outcome: | The proposed method achieves results competitive with Transformers, but is faster and scalable to large explanatory corpora. |
Chains-of-Reasoning at TextGraphs 2019 Shared Task: Reasoning over Chains of Facts for Explainable Multi-hop Inference (D19-53)
Copied to clipboard
| Challenge: | EMNLP 2019 shared task on 'Multi-hop Inference Explanation Regeneration' identifies chains of facts relevant to explain an answer to an elementary science examination question. |
| Approach: | They propose a system that identifies chains of facts relevant to explain an answer to an elementary science examination question. |
| Outcome: | The proposed system outperforms the second best system by 14.95 points on the mean average precision (MAP) metric. |
Faithful Knowledge Graph Explanations in Commonsense Question Answering (2022.emnlp-main)
Copied to clipboard
| Challenge: | Knowledge graphs are used to express explanations for the model's answer choice. |
| Approach: | They propose to use knowledge graphs to encode facts separately from the question and combine them to select an answer. |
| Outcome: | The proposed architectures can be used to express the facts used to answer a question in a graph-based explanation, but they will not include reasoning done by the transformer encoding the question, and will be incomplete. |
QED: A Framework and Dataset for Explanations in Question Answering (2021.tacl-1)
Copied to clipboard
Matthew Lamm, Jennimaria Palomaki, Chris Alberti, Daniel Andor, Eunsol Choi, Livio Baldini Soares, Michael Collins
| Challenge: | Existing question answering systems provide no explanation of reasoning that leads to answer . linguistically informed, extensible framework provides explanations in question answering . |
| Approach: | They propose a linguistically informed, extensible framework for explanations in question answering . they propose an expert-annotated dataset of QED explanations built upon a subset of the Natural Questions dataset . |
| Outcome: | The proposed framework improves the ability of untrained raters to spot errors in QA datasets. |
Learning to Explain: Datasets and Models for Identifying Valid Reasoning Chains in Multihop Question-Answering (2020.emnlp-main)
Copied to clipboard
| Challenge: | despite rapid progress in multihop question-answering, models still have trouble explaining why an answer is correct. |
| Approach: | They propose three explanation datasets in which explanations from corpus facts are annotated . they first annotate multiple candidate explanations for each answer, then use crowd-sourcing perturbations to test generalization . |
| Outcome: | The proposed datasets improve explanation quality but still behind the upper bound . the proposed dataset can be used to improve explanations using a BERT-based classifier . |