CIKQA: Learning Commonsense Inference with a Unified Knowledge-in-the-loop QA Paradigm (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing commonsense reasoning datasets target different knowledge types, modalities, and formats, but how to help machines acquire and infer over commonsensical knowledge is still unclear. |
| Approach: | They propose a commonsense reasoning benchmark to motivate commonsensing progress from two perspectives: (1) Evaluating whether models can distinguish knowledge quality by predicting if the knowledge is enough to answer the question or not. |
| Outcome: | The proposed model outperforms existing models in evaluating their generalization capabilities across tasks while demonstrating that distinguishing knowledge quality remains challenging for current models. |
Similar Papers
CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge (N19-1)
Copied to clipboard
| Challenge: | Recent work on question answering relies on factoid questions with little general knowledge. |
| Approach: | They propose a dataset to capture commonsense question answering with prior knowledge . they extract multiple-choice questions that discriminate between the source and target concepts . |
| Outcome: | The proposed dataset captures commonsense reasoning beyond associations . it obtains 56% accuracy, well below human performance, which is 89% . |
LOGICAL-COMMONSENSEQA: A Benchmark for Logical Commonsense Reasoning (2026.acl-short)
Copied to clipboard
| Challenge: | LOGICAL-COMMONSENSEQA benchmarks evaluate commonsense reasoning as logical composition over pairs of atomic statements . commonsensible reasoning is central to human cognition and a long-standing challenge in artificial intelligence and natural language understanding. |
| Approach: | They propose a benchmark that reframes commonsense reasoning as logical composition over pairs of atomic statements using plausibility-level operators. |
| Outcome: | LOGICAL-COMMONSENSEQA exposes fundamental reasoning limitations and provides a framework for advancing compositional commonsense reasoning. |
Exploring Strategies for Generalizable Commonsense Reasoning with Pre-trained Models (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent work proposes lightweight updates to improve commonsense reasoning models . fine-tuning can cause models to overfit to task-specific data and forget knowledge gained during training . |
| Approach: | They propose to use lightweight models to update pre-trained language models to learn commonsense background knowledge. |
| Outcome: | The proposed models learn from commonsense reasoning datasets, but they are overfitted and limited generalized. |
Explanations for CommonsenseQA: New Dataset and Models (2021.acl-long)
Copied to clipboard
Shourya Aggarwal, Divyanshu Mandowara, Vishwajeet Agrawal, Dinesh Khandelwal, Parag Singla, Dinesh Garg
| Challenge: | a dataset called CommonsenseQA (CQA) was recently released to advance the research on common-sense question answering (QA) |
| Approach: | They propose to retrieve and generate explanations for a given question, correct answer choice, incorrect answer choices tuple from a dataset called CommonsenseQA. |
| Outcome: | The proposed model beats baseline model by 100% in F1 score and similarity score of 61.9 . |
Generated Knowledge Prompting for Commonsense Reasoning (2022.acl-long)
Copied to clipboard
Jiacheng Liu, Alisa Liu, Ximing Lu, Sean Welleck, Peter West, Ronan Le Bras, Yejin Choi, Hannaneh Hajishirzi
| Challenge: | Existing methods for commonsense reasoning rely on high-quality knowledge, but they are often dominated by large-scale pretrained models that are fine-tuned on a target benchmark. |
| Approach: | They develop generated knowledge prompting which generates knowledge from a language model and provides it as additional input when answering a question. |
| Outcome: | The proposed method improves state-of-the-art models on four commonsense reasoning tasks. |
Fusing Context Into Knowledge Graph for Commonsense Question Answering (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to combine language modeling and knowledge graphs (KG) lack the context to provide a more precise understanding of the concepts. |
| Approach: | They propose to use external entity descriptions to provide contextual information for commonsense question answering models. |
| Outcome: | The proposed model achieves state-of-the-art among non-generative models in OpenBookQA and is the first of its kind in the field. |
Improving Unsupervised Commonsense Reasoning Using Knowledge-Enabled Natural Language Inference (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Recent methods based on pre-trained language models have shown strong supervised performance on commonsense reasoning. |
| Approach: | They propose to use a common framework to solve commonsense reasoning tasks using a dataset from NLI. |
| Outcome: | The proposed method achieves state-of-the-art unsupervised performance on two commonsense reasoning tasks. |
Beyond Surface Simplicity: Revealing Hidden Reasoning Attributes for Precise Commonsense Diagnosis (2025.acl-long)
Copied to clipboard
| Challenge: | Existing commonsense question answering benchmarks often treat these aspects in isolation, resulting in evaluation accuracy differences of up to 24.8% across different difficulty levels. |
| Approach: | They propose a framework that reveals hidden reasoning attributes behind commonsense questions by leveraging the knowledge generated during the reasoning process. |
| Outcome: | The proposed framework reveals hidden reasoning attributes behind commonsense questions by leveraging the knowledge generated during the reasoning process. |
Commonsense Reasoning for Natural Language Processing (2020.acl-tutorials)
Copied to clipboard
| Challenge: | In this tutorial, we will outline the various types of commonsense knowledge and discuss techniques to gather and represent commonsence knowledge. |
| Approach: | This tutorial will provide researchers with the critical foundations and recent advances in commonsense representation and reasoning. |
| Outcome: | This tutorial will outline the various types of commonsense and discuss techniques to gather and represent commonsence knowledge while highlighting the challenges specific to this type of knowledge (e.g., reporting bias). |
Towards Generalizable Neuro-Symbolic Systems for Commonsense Question Answering (D19-60)
Copied to clipboard
| Challenge: | Recent approaches on non-extractive commonsense QA show increased performance . attention-based injection seems to be preferable for knowledge integration . |
| Approach: | They propose to use attention-based injection to integrate knowledge into commonsense QA models. |
| Outcome: | The proposed methods show that attention-based injection is preferable for knowledge integration, and that the degree of domain overlap plays a crucial role in determining model success. |