Great~Truths~are ~Always ~Simple: A Rather Simple Knowledge Encoder for Enhancing the Commonsense Reasoning Capacity of Pre-Trained Models (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing approaches to enhance pre-trained language models (PTMs) with a knowledge-aware graph neural network (GNN) encoder that models a commonsense knowledge graph (CSKG) can't explain how external knowledge resources improve the reasoning capacity of PTMs. |
| Approach: | They propose to use relation features from CSKGs to enhance the reasoning capacity of pre-trained language models (PTMs) by encoding a commonsense knowledge graph (CSKG) |
| Outcome: | The proposed approach reduces the parameters for encoding CSKGs and improves on five benchmarks. |
Similar Papers
Improving Unsupervised Commonsense Reasoning Using Knowledge-Enabled Natural Language Inference (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Recent methods based on pre-trained language models have shown strong supervised performance on commonsense reasoning. |
| Approach: | They propose to use a common framework to solve commonsense reasoning tasks using a dataset from NLI. |
| Outcome: | The proposed method achieves state-of-the-art unsupervised performance on two commonsense reasoning tasks. |
Neural-Symbolic Commonsense Reasoner with Relation Predictors (2021.acl-short)
Copied to clipboard
| Challenge: | Existing models for commonsense reasoning are limited by their limited set of facts, rendering them unfit for reasoning over new unseen situations and events. |
| Approach: | They propose a neural-symbolic reasoner which can combine commonsense facts with large-scale dynamic CKGs to draw conclusions about ordinary situations. |
| Outcome: | The proposed model outperforms the state-of-the-art models on the task of link prediction on CKGs. |
Vera: A General-Purpose Plausibility Estimation Model for Commonsense Statements (2023.emnlp-main)
Copied to clipboard
| Challenge: | Despite the advances of language models, they still produce text that contains trivial commonsense errors. |
| Approach: | They propose a general-purpose commonsense statement verification model that learns to estimate the plausibility of declarative statements based on commonsensical knowledge. |
| Outcome: | The proposed model outperforms existing models that can be repurposed for commonsense verification, even including GPT-3.5/ChatGPT/GPT-4. |
Commonsense Reasoning for Natural Language Processing (2020.acl-tutorials)
Copied to clipboard
| Challenge: | In this tutorial, we will outline the various types of commonsense knowledge and discuss techniques to gather and represent commonsence knowledge. |
| Approach: | This tutorial will provide researchers with the critical foundations and recent advances in commonsense representation and reasoning. |
| Outcome: | This tutorial will outline the various types of commonsense and discuss techniques to gather and represent commonsence knowledge while highlighting the challenges specific to this type of knowledge (e.g., reporting bias). |
Commonsense Knowledge Transfer for Pre-trained Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in pre-trained language models have transformed the landscape of natural language processing. |
| Approach: | They propose a framework to transfer commonsense knowledge stored in a neural commonsensing model to a general-purpose pre-trained language model. |
| Outcome: | Empirical results show that the proposed framework improves the model’s performance on downstream tasks that require commonsense reasoning. |
Do Language Models Perform Generalizable Commonsense Inference? (2021.findings-acl)
Copied to clipboard
| Challenge: | Recent work has applied pretrained language models to populate commonsense knowledge graphs (CKGs) but there is a lack of understanding on their generalization to multiple CKGs, unseen relations, and novel entities. |
| Approach: | They analyze the ability of pretrained language models to perform generalizable commonsense inference in terms of knowledge capacity, transferability and induction. |
| Outcome: | The proposed models can adapt to different schemas defined by multiple CKGs but fail to generalize to new relations. |
On the Consistency of Commonsense in Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing evaluations of commonsense for large language models focus on downstream knowledge tasks, failing to probe whether LLMs truly understand and utilize knowledge or merely memorize it. |
| Approach: | They propose to automatically construct a large benchmark named CoCo which measures LLMs’ knowledge memorization, comprehension, and application and examines the consistency between these tasks. |
| Outcome: | The proposed benchmark systematically assesses LLMs’ knowledge memorization, comprehension, and application and examines the consistency between these tasks. |
I Know What You Asked: Graph Path Learning using AMR for Commonsense Reasoning (2020.coling-main)
Copied to clipboard
| Challenge: | a large amount of pre-defined commonsense knowledge is available for commonsensense reasoning . humans acquire commonsence in their lives, but machines cannot learn commonseense without assistance. |
| Approach: | They propose an AMR-ConceptNet-Pruned (ACP) graph that is pruned from a full integrated graph . they show that the ACP graph interprets the reasoning path and predicts the correct answer . |
| Outcome: | The proposed graph outperforms baseline models in the commonsenseQA task . it shows that the reasoning path can be interpreted with the relations and concepts provided by the graph . |
Structured Self-Supervised Pretraining for Commonsense Knowledge Graph Completion (2021.tacl-1)
Copied to clipboard
| Challenge: | Existing approaches focus on generating concepts that have direct and obvious relationships with existing concepts and lack an ability to generate unobvious concepts. |
| Approach: | They propose a general graph-to-paths pretraining framework that leverages high-order structures in CKGs to capture high-level relationships between concepts. |
| Outcome: | The proposed framework can capture high-order relationships between concepts in four special cases: long path, path-to-path, router, and graph-node-path. |
Modelling Commonsense Properties Using Pre-Trained Bi-Encoders (2022.coling-1)
Copied to clipboard
| Challenge: | Pre-trained language models can capture commonsense properties that are rarely expressed in text. |
| Approach: | They propose to fine-tune language models to explicitly model commonsense properties . they train separate concept and property encoders on extracted hyponym-hypernym pairs and generic sentences . |
| Outcome: | The proposed model can capture commonsense properties with higher accuracy than human models . a new study shows that the model can model commonsensence properties with much higher accuracy . |