Representation Learning of Entities and Documents from Knowledge Base Descriptions (C18-1)
Copied to clipboard
| Challenge: | Using documents extracted from Wikipedia, we train a neural network model that learns distributed representations of entities and documents directly from a knowledge base. |
| Approach: | They propose a neural network model that learns distributed representations of entities from a knowledge base. |
| Outcome: | The proposed model performs state-of-the-art on fine-grained entity typing and multiclass text classification tasks. |
Similar Papers
DOCENT: Learning Self-Supervised Entity Representations from Large Document Collections (2021.eacl-main)
Copied to clipboard
Yury Zemlyanskiy, Sudeep Gandhe, Ruining He, Bhargav Kanagal, Anirudh Ravula, Juraj Gottweis, Fei Sha, Ilya Eckstein
| Challenge: | Using pre-trained models, we learn to jointly predict words and entities from multiple text sources without any human supervision. |
| Approach: | They propose to learn rich self-supervised entity representations from large amounts of associated text. |
| Outcome: | The proposed models outperform baseline models on downstream tasks in the TV-Movies domain, and scale to very large corpora. |
Advances in Pre-Training Distributed Word Representations (L18-1)
Copied to clipboard
| Challenge: | Pre-trained word representations are a building block of many Natural Language Processing and Machine Learning applications. |
| Approach: | They propose to combine known tricks and a set of publicly available pre-trained word vector representations to train high-quality representations. |
| Outcome: | The proposed models outperform the current state of the art on a number of tasks while maintaining a high training speed to scale to massive amount of data. |
Attributed and Predictive Entity Embedding for Fine-Grained Entity Typing in Knowledge Bases (C18-1)
Copied to clipboard
| Challenge: | Existing methods for identifying semantic type of entities are incomplete even in large knowledge bases. |
| Approach: | They propose an attributed and predictive entity embedding method which can fully utilize various kinds of information comprehensively. |
| Outcome: | Experiments on two real DBpedia datasets show that the proposed method outperforms 8 state-of-the-art methods with 4.0% improvement in Mi-F1 and 5.2% improvement in Ma-F1. |
Instilling Type Knowledge in Language Models via Multi-Task QA (2022.findings-naacl)
Copied to clipboard
| Challenge: | Current methods to learn entity types rely on coarse, noisy labels . current methods rely only on text-to-text pre-training on type-centric questions . |
| Approach: | They propose to instill fine-grained type knowledge in language models by pre-training on type-centric questions. |
| Outcome: | The proposed model achieves state-of-the-art in zero-shot dialog state tracking benchmarks and can accurately infer entity types in Wikipedia articles. |
Connecting Language and Knowledge with Heterogeneous Representations for Neural Relation Extraction (N19-1)
Copied to clipboard
| Challenge: | Knowledge Bases (KBs) require constant updating to reflect changes to the world they represent. |
| Approach: | They propose a framework that unifies learning of RE and KBE models . the framework is based on a relation extraction task that uses a KB relation to a phrase . |
| Outcome: | The proposed framework unifies learning of RE and KBE models, leading to significant improvements over the state-of-the-art RE framework. |
Data-to-text Generation with Entity Modeling (P19-1)
Copied to clipboard
| Challenge: | Recent approaches to data-to-text generation have shown great promise thanks to the use of large-scale datasets and the application of neural network architectures which are trained end-to end. |
| Approach: | They propose an entity-centric neural architecture for data-to-text generation which uses hierarchical attention to create entity-specific representations which are dynamically updated. |
| Outcome: | The proposed model outperforms baselines in automatic and human evaluation on the RotoWire benchmark and a five-times larger dataset on the baseball domain. |
A Framework for Adapting Pre-Trained Language Models to Knowledge Graph Completion (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent work has demonstrated that entity representations can be extracted from pre-trained language models to develop knowledge graph completion models that are more robust to the naturally occurring sparsity found in knowledge graphs. |
| Approach: | They propose unsupervised and supervised methods to extract more informative representations from pre-trained language models to develop knowledge graph completion models. |
| Outcome: | The proposed model outperforms recent neural models in terms of performance and unsupervised processing methods. |
AutoETER: Automated Entity Type Representation for Knowledge Graph Embedding (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing knowledge graphs are incomplete whether they are constructed manually or automatically, limiting the effectiveness when exploited for downstream applications. |
| Approach: | They propose a KGE framework with an automatic type embedding mechanism which can be easily integrated into any existing KGE model. |
| Outcome: | The proposed model can model and infer all the relation patterns and complex relations compared to state-of-the-art models on four datasets. |
Topics as Entity Clusters: Entity-based Topics from Large Language Models and Graph Neural Networks (2024.lrec-main)
Copied to clipboard
| Challenge: | Topic models aim to reveal latent structures within corpus of text through term-frequency statistics over bag-of-words representations. |
| Approach: | They propose to use bimodal vector representations of entities to extract latent representations from large language models and graph neural networks trained on symbolic relations to derive the most salient aspects of these conceptual units. |
| Outcome: | The proposed approach is better suited to working with entities than state-of-the-art models. |
Representation Learning for Information Extraction from Form-like Documents (2020.acl-main)
Copied to clipboard
| Challenge: | Form-like documents like invoices, purchase orders, tax forms and insurance quotes are common in day-to-day business workflows, but current techniques for processing them largely still employ manual effort or brittle and error-prone heuristics for extraction. |
| Approach: | They propose an extraction system that uses knowledge of the types of the target fields to generate extraction candidates and a neural network architecture that learns a dense representation of each candidate based on neighboring words in the document. |
| Outcome: | The proposed system generates extraction candidates based on neighboring words in the document and is interpretable, as shown using loss cases. |