Knowledge Graph Question Answering Leaderboard: A Community Resource to Prevent a Replication Crisis (2022.lrec-1)
Copied to clipboard
| Challenge: | Knowledge Graph (KG) Question Answering (QA) is a rapidly growing field in research and industry. |
| Approach: | They propose to create a new leaderboard for any KGQA benchmark dataset as a focal point for the community. |
| Outcome: | The proposed model provides a central and open leaderboard for any KGQA benchmark dataset as a focal point for the community. |
Similar Papers
KGxBoard: Explainable and Interactive Leaderboard for Evaluation of Knowledge Graph Completion Models (2022.emnlp-demos)
Copied to clipboard
Haris Widjaja, Kiril Gashteovski, Wiem Ben Rim, Pengfei Liu, Christopher Malon, Daniel Ruffinelli, Carolin Lawrence, Graham Neubig
| Challenge: | Knowledge Graphs (KGs) store information in the form of (head, predicate, tail)-triples. |
| Approach: | They propose a framework for performing fine-grained evaluation on meaningful subsets of data. |
| Outcome: | The proposed framework tests models on meaningful subsets of the data, which would have been impossible to detect with standard averaged single-score metrics. |
Question-guided Knowledge Graph Re-scoring and Injection for Knowledge Graph Question Answering (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Knowledge graph question answering (KGQA) aims to provide factual answers to natural language questions by leveraging structured information stored in a knowledge graph. |
| Approach: | They propose a Question-guided Knowledge Graph Re-scoring method to eliminate noisy pathways for the input question, thereby focusing specifically on pertinent factual knowledge. |
| Outcome: | The proposed method eliminates noisy pathways for the input question, thereby focusing specifically on pertinent factual knowledge. |
CompleQA: Benchmarking the Impacts of Knowledge Graph Completion Methods on Question Answering (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies have focused on Knowledge Graph Completion as an end in itself, neglecting its potential impact on subsequent applications. |
| Approach: | They propose a benchmark to assess the impact of representative KGC methods on Knowledge Graph Question Answering (KGQA) they use a knowledge graph with 3 million triplets across 5 distinct domains to evaluate their results. |
| Outcome: | The proposed benchmark compares four well-known methods with two state-of-the-art systems to assess the impact of incomplete knowledge graphs on KGQA. |
Constraint-based Multi-hop Question Answering with Knowledge Graph (2022.naacl-industry)
Copied to clipboard
| Challenge: | Recent work addresses multi-hop KGQA, which requires reasoning across numerous edges of the KG. |
| Approach: | They propose to use KG embeddings to reduce KG sparsity by performing missing link prediction. |
| Outcome: | Empirical results show that the proposed method produces state-of-the-art results on three KGQA datasets. |
Enhancing Complex Reasoning in Knowledge Graph Question Answering through Query Graph Approximation (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing knowledge-grounded question answering frameworks lack essential triplets related to the questions . Existing approaches to knowledge-based QA are incomplete in the context of KGs . |
| Approach: | They propose a framework to provide answers to structured queries by leveraging Knowledge Graphs. |
| Outcome: | The proposed framework outperforms existing methods on QA tasks where KGs are incomplete . the framework is based on a set of data from a dataset of QA questions . |
Can Knowledge Graphs Make Large Language Models More Trustworthy? An Empirical Study Over Open-ended Question Answering (2025.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks for integrating Knowledge Graphs with Large Language Models focus on closed-ended tasks, leaving a gap in evaluating performance on more complex, real-world scenarios. |
| Approach: | They propose a benchmark to evaluate LLMs augmented with KGs in open-ended, real-world question answering settings. |
| Outcome: | The proposed benchmark reflects practical complexities through diverse question types and incorporates metrics to quantify both hallucination rates and reasoning improvements in LLM+KG models. |
KET-QA: A Dataset for Knowledge Enhanced Table Question Answering (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing datasets that ignore the challenge of missing knowledge in TableQA are limited in their use. |
| Approach: | They propose to use a knowledge base as the external knowledge source for TableQA and construct a dataset with fine-grained gold evidence annotation. |
| Outcome: | The proposed model achieves remarkable performance improvements on three different settings, but still lags behind the human-level performance. |
QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing question answering systems lack the ability to access relevant knowledge and reason over it. |
| Approach: | They propose a model that uses KGs to identify relevant knowledge in QA contexts and perform joint reasoning over them. |
| Outcome: | The proposed model improves on the CommonsenseQA and OpenBookQA datasets and performs interpretable and structured reasoning. |
ReaRev: Adaptive Reasoning for Question Answering over Knowledge Graphs (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Knowledge Graph Question Answering (KGQA) involves retrieving entities as answers from a Knowledge Flow using natural language queries. |
| Approach: | They propose a method to decode a question into instructions that are dense question representations used to guide the KG traversals. |
| Outcome: | The proposed method improves instruction decoding and execution by using a KG-aware information to update the initial instructions. |
Improving Multi-hop Question Answering over Knowledge Graphs using Knowledge Base Embeddings (2020.acl-main)
Copied to clipboard
| Challenge: | Existing multi-hop KGQA methods impose heuristic neighborhood limits, which often make it much harder to answer the input NL question. |
| Approach: | They propose to use knowledge Graphs (KG) to answer natural language queries over the KG. |
| Outcome: | The proposed method is particularly effective in performing multi-hop KGQA over sparse KGs. |