Incorporating Global Information in Local Attention for Knowledge Representation Learning (2021.findings-acl)
Copied to clipboard
| Challenge: | Graph Attention Networks (GATs) are a promising model that takes advantage of localized attention mechanism to perform knowledge representation learning (KRL) on graph-structure data. |
| Approach: | They propose to incorporate global information into the GAT family of models by using an attention-based global random walk algorithm. |
| Outcome: | Experimental results on KG entity prediction against the state-of-the-arts demonstrate the effectiveness of the proposed model. |
Similar Papers
Double-Branch Multi-Attention based Graph Neural Network for Knowledge Graph Completion (2023.acl-long)
Copied to clipboard
| Challenge: | Existing knowledge graph embedding methods cannot capture local and global information and are not designed well to learn representations of seen entities with sparse neighborhoods in isolated subgraphs. |
| Approach: | They propose a double-branch multi-attention based graph neural network to learn more expressive entity representations which contain rich global-local structural information. |
| Outcome: | The proposed method outperforms a general GNN-based approach for KGC. |
A2N: Attending to Neighbors for Knowledge Graph Inference (P19-1)
Copied to clipboard
| Challenge: | Existing knowledge graph completion methods learn a fixed embedding for every entity, which is suboptimal as it requires memorizing and generalizing to all possible entity relationships. |
| Approach: | They propose a method which learns query-dependent representations of entities by combining relevant neighborhood of an entity. |
| Outcome: | The proposed model performs competitively or better than existing state-of-the-art models for knowledge graph completion. |
Glocal: Incorporating Global Information in Local Convolution for Keyphrase Extraction (N19-1)
Copied to clipboard
| Challenge: | Graph Convolutional Networks (GCNs) model nodes’ local pairwise importance but lack the capability to model global relative importance in tasks where global ranking is a key component for the task. |
| Approach: | They propose to incorporate global relative importance information into the GCN family of models by using scaled node weights. |
| Outcome: | The proposed method improves keyphrase extraction by 2% and improves the baseline by 5%. |
Enhancing Contextual Word Representations Using Embedding of Neighboring Entities in Knowledge Graphs (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods for pre-trained language models lack explicit grounding in real-world entities. |
| Approach: | They propose a mechanism that integrates the structure of a KG into recent PLM architectures by generalizing the embeddings of neighboring entities. |
| Outcome: | The proposed method improves a classification task, entity typing task and language comprehension tasks. |
How Knowledge Graph and Attention Help? A Qualitative Analysis into Bag-level Relation Extraction (2021.acl-long)
Copied to clipboard
| Challenge: | Knowledge Graph (KG) and attention mechanism have been demonstrated effective in introducing and selecting useful information for weakly supervised methods. |
| Approach: | They propose a paradigm to quantitatively evaluate the effect of attention and KG on bag-level relation extraction (RE) they propose to incorporate entity prior to KG-enhanced attention to improve RE performance . |
| Outcome: | The proposed model achieves significant improvements on two real-world datasets compared with three state-of-the-art baselines. |
Learning Attention-based Embeddings for Relation Prediction in Knowledge Graphs (P19-1)
Copied to clipboard
| Challenge: | Existing knowledge graphs (KGs) are incomplete or partial information, in the form of missing relations between entities, which gives rise to the task of knowledge base completion (also known as relation prediction). |
| Approach: | They propose to capture both entity and relation features in any given neighborhood and encapsulate relation clusters and multi-hop relations in their attention-based model. |
| Outcome: | The proposed model captures both entity and relation features in any given neighborhood and also encapsulates relation clusters and multi-hop relations. |
EventKE: Event-Enhanced Knowledge Graph Embedding (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Experimental results show that events can greatly improve the quality of KG embeddings on multiple downstream tasks. |
| Approach: | They propose an event-enhanced KG embedding model that incorporates events into KGs . they first incorporate event nodes by building a heterogeneous network with event argument links . |
| Outcome: | The proposed model incorporates event nodes into the original knowledge graphs . it can be used to fuse event information into the KG embeddings on multiple tasks . |
Leveraging Local and Global Patterns for Self-Attention Networks (P19-1)
Copied to clipboard
| Challenge: | Existing approaches to integrate local and global information into self-attention networks have been criticized for overlooking neighboring information. |
| Approach: | They propose a hybrid attention mechanism to leverage local and global information . they use a gating scalar to integrate both sources of information based on local contexts . |
| Outcome: | The proposed approach improves on translation tasks and shows that the two types of contexts are complementary. |
Empowering Small-Scale Knowledge Graphs: A Strategy of Leveraging General-Purpose Knowledge Graphs for Enriched Embeddings (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing approaches to augment LLMs with Knowledge Graphs (KGs) Knowledge-intensive tasks are prone to errors and require a large amount of knowledge to be understood. |
| Approach: | They propose a framework for augmenting LLMs through Knowledge Graphs (KGs) they propose KGs can be used to enhance performance in knowledge-intensive tasks . |
| Outcome: | Experimental results show that a small domain-specific KG can benefit from a performance boost in downstream tasks when linked to a substantial general-purpose KG. |
Mitigating Attention Localization in Small Scale: Self-Attention Refinement via One-step Belief Propagation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a new framework for self-attention models is proposed to address this problem . it injects *multi-hop* relationships into the attention graph, allowing for better performance . |
| Approach: | They propose a framework that injects *multi-hop* relationships through a belief propagation process. |
| Outcome: | The proposed framework helps prevent entropy collapse in deeper layers and maintains GTD at task-appropriate levels. |