SimKGC: Simple Contrastive Knowledge Graph Completion with Pre-trained Language Models

Challenge:	Text-based methods lag behind graph embedding-based approaches for knowledge graph completion (KGC)
Approach:	They propose three types of negatives to improve contrastive learning to improve learning efficiency.
Outcome:	The proposed model outperforms embedding-based methods on several benchmark datasets.

Similar Papers

Overview

Challenge

Approach

Outcome

Prior Relational Schema Assists Effective Contrastive Learning for Inductive Knowledge Graph Completion (2024.lrec-main)

Copied to clipboard

Ruilin Luo, Jiayi Li, Jianghangfan Zhang, Jing Xiao, Yujiu Yang

Semantic Text Processing

Challenge:	Existing knowledge graphs lack robustness and incompleteness to provide link prediction.
Approach:	They propose to capture prior schema-level interactions related to relations by leveraging entity type information and introduce schema-guided negatives to bolster the efficiency of normal contrastive representation learning.
Outcome:	The proposed method achieves state-of-the-art performance on multiple established metrics across multiple datasets for link prediction.

MoCoKGC: Momentum Contrast Entity Encoding for Knowledge Graph Completion (2024.emnlp-main)

Copied to clipboard

Qingyang Li, Yanru Zhong, Yuchu Qin

Semantic Text Processing

Challenge:	Existing approaches to knowledge graph completion have not integrated the structural attributes of knowledge graphs with the textual descriptions of entities to generate robust entity encodings.
Approach:	They propose to integrate structural information from knowledge graphs with textual descriptions of entities to generate robust entity encodings.
Outcome:	The proposed model improves on the standard evaluation metric, Mean Reciprocal Rank (MRR), while surpassing the current best model on the Wikidata5M dataset.

Knowledge Context Modeling with Pre-trained Language Models for Contrastive Knowledge Graph Completion (2024.findings-acl)

Copied to clipboard

Guangqian Yang, Yi Liu, Lei Zhang, Licheng Zhang, Hongtao Xie, Zhendong Mao

Semantic Text Processing

Challenge:	Text-based knowledge graph completion methods neglect knowledge contexts in inferring process.
Approach:	They propose a framework which models the knowledge context as additional prompts with pre-trained language models for knowledge graph completion.
Outcome:	The proposed framework achieves state-of-the-art on FB15k-237, WN18RR and Wikidata5M datasets.

Improving Knowledge Graph Completion with Generative Hard Negative Mining (2023.findings-acl)

Copied to clipboard

Zile Qiao, Wei Ye, Dingyao Yu, Tong Mo, Weiping Li, Shikun Zhang

Multimodality

Challenge:	Existing methods for knowledge graph completion (KGC) use generative methods with a self-information-enhanced training strategy to generate high-quality negatives.
Approach:	They propose to leverage a sequence-to-sequence architecture to generate high-quality hard negatives from the same decoding distributions as the anchor.
Outcome:	The proposed method produces high-quality negatives with good hardness and diversity on three KGC benchmarks.

GreenKGC: A Lightweight Knowledge Graph Completion Method (2023.acl-long)

Copied to clipboard

Yun Cheng Wang, Xiou Ge, Bin Wang, C.-C. Jay Kuo

Semantic Text Processing

Challenge:	Knowledge graph completion (KGC) aims to discover missing relationships in knowledge graphs (KGs).
Approach:	They propose a modularized knowledge graph completion solution that learns embeddings for entities and relations through a score function.
Outcome:	Experimental results show that GreenKGC outperforms SOTA methods in low dimensions and even better against high-dimensional models with a much smaller model size.

Multi-perspective Improvement of Knowledge Graph Completion with Large Language Models (2024.lrec-main)

Copied to clipboard

Derong Xu, Ziheng Zhang, Zhenxi Lin, Xian Wu, Zhihong Zhu, Tong Xu, Xiangyu Zhao, Yefeng Zheng, Enhong Chen

Semantic Text Processing

Challenge:	Knowledge graph completion (KGC) is a widely used method to tackle incompleteness in knowledge graphs (KGs).
Approach:	They propose a general framework to compensate for the deficiency of contextualized knowledge by querying large language models from various perspectives.
Outcome:	The proposed framework improves knowledge graph completion (KGC) by querying large language models from various perspectives.

Does Pre-trained Language Model Actually Infer Unseen Links in Knowledge Graph Completion? (2024.naacl-long)

Copied to clipboard

Yusuke Sakai, Hidetaka Kamigaito, Katsuhiko Hayashi, Taro Watanabe

Semantic Text Processing

Challenge:	Knowledge Graph Completion (KGC) is a task that infers unseen relationships between entities . traditional embedding-based methods infer missing links using only training data . a pre-trained language model (PLM)-based KGC may be ineffective in practical applications .
Approach:	They propose to use knowledge Graph Completion (KGC) to infer unseen relationships . traditional embedding-based KGC methods infer missing links only from training data . they argue that pre-trained language models acquire inference abilities through pre-training .
Outcome:	The proposed method improves performance even though it does not use memorized knowledge.

Improving Knowledge Graph Completion with Structure-Aware Supervised Contrastive Learning (2024.emnlp-main)

Copied to clipboard

Jiashi Lin, Lifang Wang, Xinyu Lu, Zhongtian Hu, Wei Zhang, Wenxuan Lu

Semantic Text Processing

Challenge:	Existing contrastive methods focus on individual triples, overlooking the broader structural connectivities and topologies of KGs.
Approach:	They propose a new contrastive learning framework that incorporates four tasks specifically tailored to KG data: Vertex-level CL, Neighbor-level Cl, Path-levelCL, and Relation composition level CL.
Outcome:	The proposed framework achieves SOTA performance under standard supervised and low-resource settings.

Better Together: Enhancing Generative Knowledge Graph Completion with Language Models and Neighborhood Information (2023.findings-emnlp)

Copied to clipboard

Alla Chepurova, Aydar Bulatov, Yuri Kuratov, Mikhail Burtsev

Semantic Text Processing

Challenge:	Knowledge graph completion (KGC) methods are computationally intensive and impractical for large-scale KGs.
Approach:	They propose to include node neighborhoods as additional information to improve KGC methods based on language models.
Outcome:	The proposed method outperforms KGT5 and conventional methods on inductive and transductive Wikidata subsets and shows its importance.

Joint Pre-Encoding Representation and Structure Embedding for Efficient and Low-Resource Knowledge Graph Completion (2024.emnlp-main)

Copied to clipboard

Chenyu Qiu, Pengjiang Qian, Chuang Wang, Jian Yao, Li Liu, Fang Wei, Eddie Eddie

Semantic Text Processing

Challenge:	Existing knowledge graph completion models require longer training and inference times as well as increased memory usage.
Approach:	They propose to encode textual descriptions into semantic representations before training and integrate structural embedding with pre-encoded semantic description to improve model's prediction performance on 1-N relations.
Outcome:	The proposed model increases inference speed by 30x and reduces training memory by approximately 60% on the WN18RR and UMLS datasets.