Papers with QASC
Learning to Explain: Datasets and Models for Identifying Valid Reasoning Chains in Multihop Question-Answering (2020.emnlp-main)
Copied to clipboard
| Challenge: | despite rapid progress in multihop question-answering, models still have trouble explaining why an answer is correct. |
| Approach: | They propose three explanation datasets in which explanations from corpus facts are annotated . they first annotate multiple candidate explanations for each answer, then use crowd-sourcing perturbations to test generalization . |
| Outcome: | The proposed datasets improve explanation quality but still behind the upper bound . the proposed dataset can be used to improve explanations using a BERT-based classifier . |
CoSe-Co: Text Conditioned Generative CommonSense Contextualizer (2022.naacl-main)
Copied to clipboard
| Challenge: | Pre-trained language models (PTLMs) have been shown to perform well on natural language tasks. |
| Approach: | They propose a commonsense contextualizer conditioned on sentences as input to make it generically usable in tasks involving natural language text. |
| Outcome: | The proposed model improves on existing methods on CSQA, ARC, QASC and OBQA datasets. |
If You Want to Go Far Go Together: Unsupervised Joint Candidate Evidence Retrieval for Multi-hop Question Answering (2021.naacl-main)
Copied to clipboard
| Challenge: | et al. : evidence retrieval is highly dependent on partial, incorrect or no supporting knowledge. |
| Approach: | They propose a method that retrieves and reranks evidence facts jointly . they propose to account for links between sentences and coverage with the given query . |
| Outcome: | The proposed approach achieves state-of-the-art evidence retrieval performance on two multi-hop question answering datasets. |
Unsupervised Alignment-based Iterative Evidence Retrieval for Multi-hop Question Answering (2020.acl-main)
Copied to clipboard
| Challenge: | Evidence retrieval is a critical stage of question answering (QA) . Several multi-hop QA datasets have been proposed recently . |
| Approach: | They propose an unsupervised method that uses only GloVe embeddings to soft-align questions with justification sentences and an iterative process that reformulates queries focusing on terms that are not covered by existing justifications. |
| Outcome: | The proposed method outperforms all previous methods on the evidence selection task on two datasets: MultiRC and QASC. |
Flexible Generation of Natural Language Deductions (2021.emnlp-main)
Copied to clipboard
| Challenge: | Developing models that can make useful inferences from natural language premises has been a core goal in artificial intelligence since the field's early days. |
| Approach: | They propose a method for building models to generate deductive inferences from diverse natural language inputs without direct human supervision. |
| Outcome: | The proposed model is more accurate and flexible than baseline systems. |
Question Difficulty Estimation for Large Language Models via Answer Plausibility Scoring (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to estimate question difficulty rely on readability formulas, retrieval-based signals, or popularity statistics. |
| Approach: | They propose a method that estimates question difficulty by computing the entropy of plausibility scores over candidate answers. |
| Outcome: | The proposed method outperforms baselines across four QA datasets and shows strong robustness across hyperparameter variations and question types. |
NSF-CoT: Neuro-Symbolic Formal Verification of Chain-of-Thought Faithfulness in Contextual Question Answering (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing faithfulness checks only test whether changing the written chain changes the answer, without verifying whether the steps are truly supported by the given evidence. |
| Approach: | They propose a method that checks CoT faithfulness step by step for contextual question answering. |
| Outcome: | The proposed method outperforms causal mediation, perturbation probes, and behavioral monitoring in OpenBookQA, QASC, and HotpotQA. |
Reasoning or a Semblance of it? A Diagnostic Study of Transitive Reasoning in LLMs (2024.emnlp-main)
Copied to clipboard
| Challenge: | Evaluating Large Language Models (LLMs) on reasoning benchmarks demonstrates their ability to solve compositional questions. |
| Approach: | They investigate the transitive reasoning capabilities of two distinct LLM architectures, LLaMA 2 and Flan-T5, by manipulating facts within two compositional datasets: QASC and Bamboogle. |
| Outcome: | The proposed models leverage word/phrase overlaps across sections of test input, models’ inherent knowledge during pre-training or fine-tuning, and names of entities. |
Distinguish Before Answer: Generating Contrastive Explanation as Knowledge for Commonsense Question Answering (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing knowledge-enhanced methods have trouble obtaining knowledge from different knowledge bases . a concept-centric model can be used to generate a contrastive explanation for QA tasks . |
| Approach: | They propose a Concept-centric Prompt-bAsed Contrastive Explanation Generation model which converts obtained symbolic knowledge into the contrastive explanation for better distinguishing the differences among given candidates. |
| Outcome: | The proposed model achieves new SOTA on CSQA, QASC, and OBQA. |
Faithful and Robust LLM-Driven Theorem Proving for NLI Explanations (2025.acl-long)
Copied to clipboard
| Challenge: | Recent work has shown that the interaction of large language models (LLMs) with theorem provers (TPs) can help verify and improve the validity of NLI explanations. |
| Approach: | They propose to use logical expressions to guide LLMs in generating structured proof sketches and to use them to improve their accuracy. |
| Outcome: | The proposed strategies improve autoformalisation, syntactic errors and explanation refinement over the state-of-the-art model. |