GraphEval36K: Benchmarking Coding and Reasoning Capabilities of Large Language Models on Graph Datasets (2025.findings-naacl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated significant capabilities in processing and understanding text data. |
| Approach: | They propose a structure-based instruction-based method to enhance LLM performance on complex graph tasks. |
| Outcome: | The proposed framework outperforms open-source models on graph problem-solving, but the gap is narrowing. |
Similar Papers
Exploring Graph Learning Tasks with Pure LLMs: A Comprehensive Benchmark and Investigation (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent studies focus on performance benchmarks without fully comparing LLMs to graph learning models. |
| Approach: | They evaluate off-the-shelf and instruction-tuned graph learning models across a variety of scenarios. |
| Outcome: | The proposed models outperform traditional graph learning models in few-shot settings, the authors show . their models out perform models with instruction tuning, and they show excellent generalization and robustness. |
Demystifying the Power of Large Language Models in Graph Generation (2025.findings-naacl)
Copied to clipboard
Yu Wang, Ryan A. Rossi, Namyong Park, Nesreen K. Ahmed, Danai Koutra, Franck Dernoncourt, Tyler Derr
| Challenge: | Large Language Models (LLMs) have been used for graph discriminative tasks, but their potential for graph structure generation remains unexplored. |
| Approach: | They propose to use LLMs to generate graphs that optimize network properties by injecting domain expertise from network science into the code. |
| Outcome: | The proposed model generates graphs satisfying each property in different domains and compares it with established graph generative models across multiple domains. |
Graph-Assisted Large Language Models: A Perspective on Mitigating Intrinsic Limitations (2026.findings-acl)
Copied to clipboard
Haitong Luo, Fali Wang, Weiyao Zhang, Xianren Zhang, Zhiwei Zhang, Tianxiang Zhao, Minhua Lin, Jiahao Zhang, Hui Liu, Xianfeng Tang, Qi He, Suhang Wang, Xuying Meng, Yujun Zhang
| Challenge: | Large language models exhibit intrinsic limitations such as knowledge cutoff, single-threaded reasoning that hinders finer-grained branch and aggregation, and rigid collaboration mechanisms that struggle to coordinate specialized capabilities. |
| Approach: | They propose a taxonomy spanning *Graph-Assisted Knowledge Augmentation*, *Graph Assisted Reasoning and Planning*, and *Graphed LLM Collaboration*. |
| Outcome: | The proposed models show that graphs can augment and correct LLMs and support dynamic coordination among experts and agents in collaborative settings. |
Are LLMs Truly Graph-Savvy? A Comprehensive Evaluation of Graph Generation (2025.acl-srw)
Copied to clipboard
| Challenge: | Large language models have demonstrated impressive capabilities across diverse tasks . however, their ability to generate valid graph structures remains underexplored . |
| Approach: | They evaluate large language models on five specialized graph generation tasks . they also test the models using 3 different prompt types: direct, iterative feedback, and program-augmented. |
| Outcome: | The proposed models solve twice as many tasks as general-purpose models, compared to their general-usage peers. |
KG-GPT: A General Framework for Reasoning on Knowledge Graphs Using Large Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Using large language models for complex reasoning tasks on knowledge graphs remains unexplored. |
| Approach: | They propose a multi-purpose framework leveraging large language models for complex reasoning tasks on knowledge graphs. |
| Outcome: | The proposed framework outperforms fully-supervised models in KG-based fact verification and KGQA benchmarks. |
Evaluating LLMs on Large-Scale Graph Property Estimation via Random Walks (2026.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have improved reasoning abilities but are limited due to limited context length. |
| Approach: | They propose a large graph benchmark dataset and propose four tasks to evaluate LLMs' reasoning abilities. |
| Outcome: | The proposed tasks evaluate the reasoning abilities of LLMs on a large graph benchmark dataset. |
Each graph is a new language: Graph Learning with LLMs (2025.findings-acl)
Copied to clipboard
| Challenge: | Natural language is used to describe graphs, but graph descriptions become verbose and only relying on attribute embeddings limits LLM’s ability to capture adequate graph structural information. |
| Approach: | They propose a graph-defined language for large language model that translates the graph into a corpus instead of graph descriptions and pre-trains LLMs on this corpus to adequately understand the graph. |
| Outcome: | Experiments on five datasets show that the proposed framework outperforms description-based and embedding-based baselines by efficiently modeling different orders of neighbors. |
A Graph Talks, But Who’s Listening? Rethinking Evaluations for Graph-Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks for Graph-Language Models (GLMs) do not assess true multimodal integration. |
| Approach: | They propose a benchmark to evaluate multimodal reasoning over graph topology and textual semantics. |
| Outcome: | The proposed benchmarks show that strong performance is achievable using textual or structural features in isolation, bypassing the need for joint reasoning. |
Can Large Language Models Tackle Graph Partitioning? (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have remarkable capabilities in understanding complex tasks, but they can only handle graph partitioning tasks that require global perception abilities. |
| Approach: | They propose a pipeline for coarsening, reasoning, and refining to enable LLMs to perform graph partitioning on small-scale graphs. |
| Outcome: | The proposed pipeline can handle graph partitioning tasks on small graphs with coarsening, reasoning, and refining. |
LLMs Underperform Graph-Based Parsers on Supervised Relation Extraction for Complex Graphs (2026.acl-short)
Copied to clipboard
| Challenge: | Relation extraction is a core NLP task which involves extracting [head, relation, dependent] RDF triples from text. |
| Approach: | They evaluate four large language models against a graph-based parser on six relation extraction datasets with sentence graphs of varying sizes and complexities. |
| Outcome: | The graph-based parser outperforms the LLMs on six relation extraction datasets with sentence graphs of varying sizes and complexities. |