Coconut: Contextualized Commonsense Unified Transformers for Graph-Based Commonsense Augmentation of Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing studies show that pre-trained language models lack commonsense knowledge . |
| Approach: | They propose a contextualized knowledge prompting scheme to guide the contextualization of structured commonsense knowledge based on large language models. |
| Outcome: | The proposed approach outperforms the state-of-the-art technique by an average of 5.8%. |
Similar Papers
On the Consistency of Commonsense in Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing evaluations of commonsense for large language models focus on downstream knowledge tasks, failing to probe whether LLMs truly understand and utilize knowledge or merely memorize it. |
| Approach: | They propose to automatically construct a large benchmark named CoCo which measures LLMs’ knowledge memorization, comprehension, and application and examines the consistency between these tasks. |
| Outcome: | The proposed benchmark systematically assesses LLMs’ knowledge memorization, comprehension, and application and examines the consistency between these tasks. |
Cracking the Contextual Commonsense Code: Understanding Commonsense Reasoning Aptitude of Deep Contextual Representations (D19-60)
Copied to clipboard
| Challenge: | Pretrained deep contextual representations have advanced the state-of-the-art on various commonsense NLP tasks, but we lack a concrete understanding of their capabilities. |
| Approach: | They investigate BERT's ability to encode various commonsense features in its embedding space, but are still deficient in many areas. |
| Outcome: | The proposed model improves performance on a downstream commonsense reasoning task while using minimal data. |
Improving Unsupervised Commonsense Reasoning Using Knowledge-Enabled Natural Language Inference (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Recent methods based on pre-trained language models have shown strong supervised performance on commonsense reasoning. |
| Approach: | They propose to use a common framework to solve commonsense reasoning tasks using a dataset from NLI. |
| Outcome: | The proposed method achieves state-of-the-art unsupervised performance on two commonsense reasoning tasks. |
COMET: Commonsense Transformers for Automatic Knowledge Graph Construction (P19-1)
Copied to clipboard
| Challenge: | Existing studies on commonsense knowledge base construction only store loosely structured open-text descriptions of knowledge. |
| Approach: | They propose a commonsense knowledge base construction model that generates rich commonsensense descriptions in natural language. |
| Outcome: | The proposed models can generate rich and diverse commonsense descriptions in natural language. |
CAT: A Contextualized Conceptualization and Instantiation Framework for Commonsense Reasoning (2023.acl-long)
Copied to clipboard
| Challenge: | HKUST-KnowComp proposes a framework for commonsense reasoning that can be used to conceptualize commonsence knowledge bases at scale. |
| Approach: | They propose a framework that integrates event conceptualization and instantiation to conceptualize commonsense knowledge bases at scale. |
| Outcome: | The proposed framework achieves state-of-the-art on two conceptualization tasks and the acquired abstract commonsense knowledge significantly improves commonsence inference modeling. |
Generative Data Augmentation for Commonsense Reasoning (2020.findings-emnlp)
Copied to clipboard
Yiben Yang, Chaitanya Malaviya, Jared Fernandez, Swabha Swayamdipta, Ronan Le Bras, Ji-Ping Wang, Chandra Bhagavatula, Yejin Choi, Doug Downey
| Challenge: | Recent advances in commonsense reasoning depend on large-scale human-authored training data. |
| Approach: | They propose a generative data augmentation technique that augments human-authored training data by using pretrained language models. |
| Outcome: | The proposed technique outperforms existing methods on commonsense reasoning benchmarks and enhances out-of-distribution generalization. |
Jeff Da at COIN - Shared Task: BIG MOOD: Relating Transformers to Explicit Commonsense Knowledge (D19-60)
Copied to clipboard
| Challenge: | Recent studies show that large-scale pre-training models can be effective for large datasets. |
| Approach: | They propose a method of integrating contextual embeddings with commonsense graph embeddINGs by preprocessing knowledge bases and aligning tokens between misaligned tokenization methods. |
| Outcome: | The proposed method achieves higher accuracy than BERT and scores highest without pretraining. |
Enhancing Natural Language Representation with Large-Scale Out-of-Domain Commonsense (2022.findings-acl)
Copied to clipboard
| Challenge: | Using commonsense in text understanding tasks can cause catastrophic forgetting due to domain discrepancy . previous methods of using textual descriptions as extra input information cannot apply to large-scale commonsensing. |
| Approach: | They propose to use out-of-domain commonsense to enhance text representation . they propose to integrate commonsensense descriptions into large-scale models . |
| Outcome: | The proposed model can integrate commonsense descriptions and enhance them to the target text representation without pre-training on large-scale unsupervised corpora. |
Do Language Models Perform Generalizable Commonsense Inference? (2021.findings-acl)
Copied to clipboard
| Challenge: | Recent work has applied pretrained language models to populate commonsense knowledge graphs (CKGs) but there is a lack of understanding on their generalization to multiple CKGs, unseen relations, and novel entities. |
| Approach: | They analyze the ability of pretrained language models to perform generalizable commonsense inference in terms of knowledge capacity, transferability and induction. |
| Outcome: | The proposed models can adapt to different schemas defined by multiple CKGs but fail to generalize to new relations. |
CoCoLM: Complex Commonsense Enhanced Language Model with Discourse Relations (2022.findings-acl)
Copied to clipboard
| Challenge: | Large-scale pre-trained language models have demonstrated strong knowledge representation ability, but struggle with complex commonsense knowledge that involves multiple eventualities. |
| Approach: | They propose to help pre-trained language models better incorporate complex commonsense knowledge that involves multiple eventualities. |
| Outcome: | The proposed model can learn to use the memorized knowledge for different tasks and achieve outstanding performance on many downstream natural language processing (NLP) tasks. |