Barack’s Wife Hillary: Using Knowledge Graphs for Fact-Aware Language Modeling (P19-1)
Copied to clipboard
| Challenge: | Existing language models are only capable of remembering facts seen at training time, and have difficulty recalling them. |
| Approach: | They introduce a knowledge graph language model with mechanisms for selecting and copying facts from a Knowledge graph that are relevant to the context. |
| Outcome: | The proposed model outperforms a baseline language model in generating factual knowledge and generating sentences that require factual information. |
Similar Papers
Knowledge Graph Based Synthetic Corpus Generation for Knowledge-Enhanced Language Model Pre-training (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing work on data-to-text generation focused on domain-specific benchmark datasets. |
| Approach: | They use a KG-Wikipedia text aligned corpus to verbalize the entire English Wikidata KG . they show that this approach can be used to integrate structured KGs and natural language corpora . |
| Outcome: | The proposed method improves on open domain QA and the LAMA knowledge probe. |
Adapters for Enhanced Modeling of Multilingual Knowledge and Text (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models learn facts from text corpora, but knowledge graphs contain facts in an explicit triple format, restricting their research and application. |
| Approach: | They propose to enhance multilingual language models with knowledge from multilingual knowledge graphs . they propose to use cross-lingual entity alignment and facts from MLKGs to improve performance . |
| Outcome: | The proposed model improves MLLMs with cross-lingual entity alignment and facts from multilingual knowledge graphs for many languages while maintaining performance on other general language tasks. |
Graph Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Language Models (LMs) are the workhorses of NLP, but their interplay with structured knowledge graphs (KGs) is still actively researched. |
| Approach: | They propose a Graph Language Model (GLM) that integrates the strengths of both approaches and mitigates their weaknesses. |
| Outcome: | Empirical evaluations show that the proposed model surpasses both LM- and GNN-based baselines in supervised and zero-shot setting, demonstrating their versatility. |
Retrieval and Reasoning on KGs: Integrate Knowledge Graphs into Large Language Models for Complex Question Answering (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have performed impressively in various NLP tasks, but their inherent hallucination phenomena severely challenge their credibility in complex reasoning. |
| Approach: | They propose to integrate explainable Knowledge Graphs (KGs) with LLMs to alleviate hallucinations . they construct subgraphs to enhance the retrieval capabilities of KGs via CoT reasoning. |
| Outcome: | Extensive experiments on two KGQA datasets show that the proposed model achieves convincing performance compared to strong baselines. |
QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing question answering systems lack the ability to access relevant knowledge and reason over it. |
| Approach: | They propose a model that uses KGs to identify relevant knowledge in QA contexts and perform joint reasoning over them. |
| Outcome: | The proposed model improves on the CommonsenseQA and OpenBookQA datasets and performs interpretable and structured reasoning. |
Digest the Knowledge: Large Language Models empowered Message Passing for Knowledge Graph Question Answering (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods to augment large language models (LLMs) with external knowledge are unorganized and unorganized. |
| Approach: | They propose a method that learns a concise facts graph and encodes it into multi-level lists of texts to augment LLMs. |
| Outcome: | The proposed method improves on all 5 knowledge graph question answering datasets and offers human-level semantic explainability. |
Few-shot Knowledge Graph-to-Text Generation with Pretrained Language Models (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing models for KG-to-text generation are based on pretrained language models. |
| Approach: | They propose to automatically generate a text that describes the facts in knowledge graph (KG) they leverage the excellent capacities of pretrained language models (PLMs) in language understanding and generation. |
| Outcome: | The proposed model outperforms all comparison methods on fully-supervised and fewshot settings. |
What Has Been Enhanced in my Knowledge-Enhanced Language Model? (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing knowledge integration methods such as linear probes and prompts have key limitations in answering these questions. |
| Approach: | They propose a new probe model which integrates external knowledge from knowledge graphs into pretrained language models (LMs) ERNIE and K-Adapter are proposed as KI methods . |
| Outcome: | The proposed model interprets two well-known KELMs using graph attention on the corresponding knowledge graph for interpretation. |
Knowledge Graph-Enhanced Large Language Models via Path Selection (2024.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown unprecedented performance in various real-world applications, but they are known to generate factually inaccurate outputs. |
| Approach: | They propose a framework to integrate external knowledge extracted from Knowledge Graphs (KGs) they propose to generate scores for knowledge paths with input texts via latent semantic matching. |
| Outcome: | Experiments on real-world datasets validate the effectiveness of a framework to extract knowledge from Knowledge Graphs (KGs) incorporating external knowledge has become a promising strategy to improve the factual accuracy of LLM-generated outputs. |
GAP: A Graph-aware Language Model Framework for Knowledge Graph-to-Text Generation (2022.coling-1)
Copied to clipboard
| Challenge: | Recent improvements in KG-to-text generation are due to additional pre-training tasks . these tasks require extensive computational resources while only suggesting marginal improvements. |
| Approach: | They propose a mask structure to capture neighborhood information and a type encoder that adds a bias to the graph-attention weights depending on the connection type. |
| Outcome: | The proposed model outperforms state-of-the-art models while requiring no additional pre-training tasks. |