Efficient Entity Embedding Construction from Type Knowledge for BERT (2022.findings-aacl)
Copied to clipboard
| Challenge: | Existing work has shown advantages of incorporating knowledge graphs (KGs) into BERT for various NLP tasks. |
| Approach: | They propose to integrate knowledge graphs into BERT to train entity embeddings to include rich information of factual knowledge. |
| Outcome: | The proposed models perform very well when combined with context. |
Similar Papers
E-BERT: Efficient-Yet-Effective Entity Embeddings for BERT (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to enhance BERT with factual knowledge about entities require no additional pretraining and no changes to the encoder itself. |
| Approach: | They propose a way to inject factual knowledge into the pretrained BERT model by aligning Wikipedia2Vec entity vectors with BERT's native wordpiece vector space and feeding the aligned entity vector into BERT as if they were wordpieces. |
| Outcome: | The proposed version outperforms baseline models on unsupervised question answering, supervised relation classification and entity linking tasks. |
Enhancing Contextual Word Representations Using Embedding of Neighboring Entities in Knowledge Graphs (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods for pre-trained language models lack explicit grounding in real-world entities. |
| Approach: | They propose a mechanism that integrates the structure of a KG into recent PLM architectures by generalizing the embeddings of neighboring entities. |
| Outcome: | The proposed method improves a classification task, entity typing task and language comprehension tasks. |
Connecting Embeddings for Knowledge Graph Entity Typing (2020.acl-main)
Copied to clipboard
| Challenge: | Existing knowledge graphs suffer from incompleteness and miss important facts, jeopardizing their usefulness in downstream tasks such as question answering. |
| Approach: | They propose a method which is trained by utilizing local typing knowledge from existing entity type assertions and global triple knowledge in KGs. |
| Outcome: | The proposed model favors inferences that agree with both entity type instances and triple knowledge in KGs. |
Fusing Label Embedding into BERT: An Efficient Improvement for Text Classification (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to improve text classification performance of pre-trained models have been used to improve their performance. |
| Approach: | They propose a method for improving BERT's performance by using a label embedding technique while keeping almost the same computational cost. |
| Outcome: | The proposed method improves BERT's performance on six text classification benchmark datasets while keeping almost the same computational cost. |
Attributed and Predictive Entity Embedding for Fine-Grained Entity Typing in Knowledge Bases (C18-1)
Copied to clipboard
| Challenge: | Existing methods for identifying semantic type of entities are incomplete even in large knowledge bases. |
| Approach: | They propose an attributed and predictive entity embedding method which can fully utilize various kinds of information comprehensively. |
| Outcome: | Experiments on two real DBpedia datasets show that the proposed method outperforms 8 state-of-the-art methods with 4.0% improvement in Mi-F1 and 5.2% improvement in Ma-F1. |
Interpretable Entity Representations through Large-Scale Typing (2020.findings-emnlp)
Copied to clipboard
| Challenge: | In traditional methods for natural language processing, entities are embedded in dense vector spaces with pre-trained models. |
| Approach: | They propose an approach to creating entity representations that are human readable and achieve high performance on entity-related tasks out of the box. |
| Outcome: | The proposed representations are vectors whose values correspond to posterior probabilities over fine-grained entity types, indicating the confidence of a typing model’s decision that the entity belongs to the corresponding type. |
Pretrained Knowledge Base Embeddings for improved Sentential Relation Extraction (2022.acl-srw)
Copied to clipboard
| Challenge: | Existing models that perform explicit on-task training of graph embeddings are inadequate. |
| Approach: | They propose to combine pretrained knowledge base graph embeddings with transformer based language models to improve performance on sentential Relation Extraction task. |
| Outcome: | The proposed model outperforms state-of-the-art models on the sentential Relation Extraction task. |
Static Embeddings as Efficient Knowledge Bases? (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent research investigates factual knowledge stored in large pretrained language models . masked sentences such as “Paris is the capital of [MASK]” are used as probes . |
| Approach: | They use masked sentences to test whether a language model can capture factual knowledge . they show that static embeddings perform better than PLMs when restricted to a candidate set . |
| Outcome: | The results show that static embeddings perform better than PLMs when restricted to a candidate set . |
Embeddings in Natural Language Processing (2020.coling-tutorials)
Copied to clipboard
| Challenge: | Embeddings have been a key topic of interest in NLP for the past decade . a quick warm-up introduction to NLP and why it is important to have a semantic comprehension of texts . |
| Approach: | This tutorial will provide a high-level synthesis of the main embedding techniques in NLP . it will start with word embedds and then move to other types of embeddable vectors . |
| Outcome: | This tutorial will provide a high-level synthesis of the main embedding techniques in NLP . it will start with word embedds and move to other types of embeddable representations . |
Knowledge Graph Alignment with Entity-Pair Embedding (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for Knowledge Graph (KG) alignment are not satisfactory. |
| Approach: | They propose a method that directly learns embeddings of entity-pairs for KG alignment. |
| Outcome: | The proposed approach can achieve state-of-the-art on five real-world datasets. |