Papers with ATOMIC
ZhuJiu-Knowledge: A Fairer Platform for Evaluating Multiple Knowledge Types in Large Language Models (2024.naacl-demo)
Copied to clipboard
| Challenge: | evaluating the knowledge of large language models (LLMs) is crucial, and rapid advancement in large language modeling has heightened the importance of model evaluations. |
| Approach: | They propose a fairer benchmark for evaluating multiple knowledge types of LLMs by focusing on commonsense knowledge, world knowledge, and language knowledge. |
| Outcome: | The proposed framework evaluates 14 current mainstream LLMs and provides a detailed discussion and analysis of their results. |
Identify, Align, and Integrate: Matching Knowledge Graphs to Commonsense Reasoning Tasks (2021.eacl-main)
Copied to clipboard
| Challenge: | Empirically, we investigate KG matches for the SocialIQA, Physical IQA, and MCScript2.0 datasets with 3 diverse KGs: ATOMIC (SIQA), ConceptNet (Speer et al., 2017), and an automatically constructed instructional KG based on WikiHow (Ostermann e., 2019b). |
| Approach: | They propose a method to assess how well a candidate KG can fill in knowledge gaps for a given task by using commonsense probes. |
| Outcome: | Empirically, we show that the proposed KG-to-task match is a good match for socialIQA, physical IQA, and MCScript2.0 datasets with 3 diverse KGs: ATOMIC, ConceptNet, and an instructional KG based on WikiHow. |
COMET: Commonsense Transformers for Automatic Knowledge Graph Construction (P19-1)
Copied to clipboard
| Challenge: | Existing studies on commonsense knowledge base construction only store loosely structured open-text descriptions of knowledge. |
| Approach: | They propose a commonsense knowledge base construction model that generates rich commonsensense descriptions in natural language. |
| Outcome: | The proposed models can generate rich and diverse commonsense descriptions in natural language. |
Dense-ATOMIC: Towards Densely-connected ATOMIC with High Knowledge Coverage and Massive Multi-hop Paths (2023.acl-long)
Copied to clipboard
| Challenge: | ATOMIC is a large-scale commonsense knowledge graph (CSKG) containing everyday if-then knowledge triplets, i.e., head event, relation, tail event. |
| Approach: | They propose a CSKG completion method called Rel-CSKGC to predict the relation given the head event and tail event of a triplet and train a model based on existing triplets. |
| Outcome: | The proposed method is based on existing triplets and can be used to complete the missing links in ATOMIC. |