BertNet: Harvesting Knowledge Graphs with Arbitrary Relations from Pretrained Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to construct knowledge graphs are limited to a small set of relations due to manual cost or restrictions in text corpus. |
| Approach: | They propose to automatically construct knowledge graphs (KGs) of diverse new relations from pretrained language models that accept knowledge queries with prompts. |
| Outcome: | The proposed framework extracts knowledge of over 400 new relations from pretrained language models, including RoBERTaNet, with minimal input of a relation definition and a few shot of example entity pairs. |
Similar Papers
Pretrain-KGE: Learning Knowledge Representation from Pretrained Language Models (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing knowledge graph embedding models suffer from limited knowledge representation due to sparse and noisy dataset annotations. |
| Approach: | They propose to use pretrained language models to enhance knowledge representation by leveraging world knowledge from pretrained models. |
| Outcome: | Extensive experiments show that the proposed framework can improve results over existing models. |
Multi-Task Learning for Knowledge Graph Completion with Pre-trained Language Models (2020.coling-main)
Copied to clipboard
| Challenge: | Existing knowledge graph completion methods are lacking in ranking metrics such as Hits@k . despite the high performance, the proposed method is still behind state-of-the-art models. |
| Approach: | They propose a multi-task learning method that integrates relational and relevance ranking tasks with target link prediction to improve ranking performance. |
| Outcome: | The proposed method improves ranking performance but still behind state-of-the-art models in Hits@k and Mean Rank metrics. |
Pretrained Knowledge Base Embeddings for improved Sentential Relation Extraction (2022.acl-srw)
Copied to clipboard
| Challenge: | Existing models that perform explicit on-task training of graph embeddings are inadequate. |
| Approach: | They propose to combine pretrained knowledge base graph embeddings with transformer based language models to improve performance on sentential Relation Extraction task. |
| Outcome: | The proposed model outperforms state-of-the-art models on the sentential Relation Extraction task. |
Generating Domain-Specific Knowledge Graphs from Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have shown impressive world knowledge across different benchmarks and domains but their knowledge is inconveniently scattered across their billions of parameters. |
| Approach: | They propose a prompt-based method to extract knowledge solely from LLMs’ parameters to construct domain-specific KGs by a schema-based process. |
| Outcome: | The proposed method generates large domain-specific KGs containing tens of thousands of entities and relations, and then evaluates against Wikidata, an open-source human-created KG. |
Language Models as Knowledge Bases? (D19-1)
Copied to clipboard
Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, Alexander Miller
| Challenge: | Recent advances in pretraining language models on large textual corpora led to a surge of improvements for downstream NLP tasks. |
| Approach: | They present a method for pretraining language models on large textual corpora . they find that they can store relational knowledge and answer queries structured as "fill-in-the-blank" queries. |
| Outcome: | The proposed language models can recall factual knowledge without fine-tuning without fine tuning . the proposed models can answer queries structured as "fill-in-the-blank" cloze statements . |
KLMo: Knowledge Graph Enhanced Pretrained Language Model with Fine-Grained Relationships (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing knowledge-enhanced pretrained language models focus on entity information and ignore fine-grained relationships between entities. |
| Approach: | They propose to incorporate KG into the language learning process to obtain a KG-enhanced pretrained Language Model. |
| Outcome: | The proposed model improves on several knowledge-driven tasks, such as entity typing and relation classification, compared with the state-of-the-art knowledge-enhanced PLMs. |
Retrieval and Reasoning on KGs: Integrate Knowledge Graphs into Large Language Models for Complex Question Answering (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have performed impressively in various NLP tasks, but their inherent hallucination phenomena severely challenge their credibility in complex reasoning. |
| Approach: | They propose to integrate explainable Knowledge Graphs (KGs) with LLMs to alleviate hallucinations . they construct subgraphs to enhance the retrieval capabilities of KGs via CoT reasoning. |
| Outcome: | Extensive experiments on two KGQA datasets show that the proposed model achieves convincing performance compared to strong baselines. |
Large-Scale Relation Learning for Question Answering over Knowledge Bases with Pre-trained Language Models (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing KBQA methods focus on the natural language but ignore textual information carried by the nodes and edges. |
| Approach: | They propose to perform relation extraction, relation matching, and relation reasoning tasks to align the natural language expressions to the relations in the KB and reason over the missing connections. |
| Outcome: | Experiments on WebQSP show that the proposed model outperforms baselines even when the KB is incomplete. |
Does Pre-trained Language Model Actually Infer Unseen Links in Knowledge Graph Completion? (2024.naacl-long)
Copied to clipboard
| Challenge: | Knowledge Graph Completion (KGC) is a task that infers unseen relationships between entities . traditional embedding-based methods infer missing links using only training data . a pre-trained language model (PLM)-based KGC may be ineffective in practical applications . |
| Approach: | They propose to use knowledge Graph Completion (KGC) to infer unseen relationships . traditional embedding-based KGC methods infer missing links only from training data . they argue that pre-trained language models acquire inference abilities through pre-training . |
| Outcome: | The proposed method improves performance even though it does not use memorized knowledge. |
Mind the Labels: Describing Relations in Knowledge Graphs With Pretrained Models (2023.eacl-main)
Copied to clipboard
| Challenge: | Pretrained language models (PLMs) for data-to-text generation produce inaccurate outputs if labels are ambiguous or incomplete, which is often the case in D2T datasets. |
| Approach: | They propose to use a dataset to descib a relation between two entities using relation labels to train pretrained language models. |
| Outcome: | The proposed models are robust to generalizing to out-of-domain domains on a dataset for descibing a relation between two entities. |