EVGeoQA: Benchmarking LLMs on Dynamic, Multi-Objective Geo-Spatial Exploration (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing geo-spatial question answering benchmarks focus on static retrieval, failing to capture the complexity of real-world planning that involves dynamic user locations and compound constraints. |
| Approach: | They propose a new benchmark for Large Language Models that integrates location-anchored and dual-objective queries with a user's real-time coordinates. |
| Outcome: | The proposed model can summarize historical exploration trajectories to enhance exploration efficiency. |
Similar Papers
UrbanGeoEval: A City-Scale Benchmark for Evaluating Large Language Models in Geospatial Reasoning (2026.acl-long)
Copied to clipboard
| Challenge: | Extensive experiments on 18 widely used LLMs uncover critical insights: (1) models exhibit severe geographic biases and resolution gaps; (2) failures in complex multi-hop tasks stem from brittle foundational spatial skills rather than high-level logic deficits. |
| Approach: | They propose a dual-module framework that disentangles factual recall and spatial logic from the model's real capabilities in urban environments. |
| Outcome: | Extensive tests on 18 widely used LLMs reveal that models exhibit severe geographic biases and resolution gaps, and failures in complex multi-hop tasks often stem from brittle foundational spatial skills rather than high-level logic deficits. |
GER-LLM: Efficient and Effective Geospatial Entity Resolution with Large Language Model (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for integrating spatial data from diverse sources are limited by their reliance on large amounts of training data and their inability to incorporate commonsense knowledge. |
| Approach: | They propose a framework that integrates large language models into the GER pipeline. |
| Outcome: | The proposed framework improves on real-world geospatial datasets and shows that it is more efficient than state-of-the-art methods. |
CityEQA: A Hierarchical LLM Agent on Embodied Question Answering Benchmark in City Space (2025.emnlp-main)
Copied to clipboard
Yong Zhao, Kai Xu, Zhengqiu Zhu, Yue Hu, Zhiheng Zheng, Yingfeng Chen, Yatai Ji, Chen Gao, Yong Li, Jincai Huang
| Challenge: | Embodied Question Answering (EQA) tasks are primarily focused on indoor environments, leaving the complexities of urban settings unexplored. |
| Approach: | They propose a task where an embodied agent answers open-vocabulary questions in dynamic city spaces. |
| Outcome: | The proposed agent achieves 60.7% of human-level answering accuracy compared to baselines . the proposed agent outperforms existing agents in open-ended city spaces . |
Automatic Generation of a Compositional QA Benchmark for Geospatial Reasoning under Spatial and Entity Constraints (2026.eacl-srw)
Copied to clipboard
| Challenge: | Recent advances in large language models have enhanced their ability to perform reasoning tasks that integrate linguistic, visual, and factual information. |
| Approach: | They propose a method for constructing compositional geographic question answering datasets that jointly consider spatial and entity constraints. |
| Outcome: | The proposed method performs well on questions involving rich entity grounding, but its accuracy drops on quantitative spatial reasoning questions. |
CompassLLM: A Multi-Agent Approach toward Geo-Spatial Reasoning for Popular Path Query (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing algorithms and machine learning methods require model training, parameter tuning, and retraining when accommodating data updates. |
| Approach: | They propose a multi-agent framework that leverages the reasoning capabilities of Large Language Models into the geo-spatial domain to solve the popular path query. |
| Outcome: | Experiments on real and synthetic datasets show that CompassLLM performs better than existing models while being cost-effective. |
Can Multimodal Large Language Models Understand Spatial Relations? (2025.acl-long)
Copied to clipboard
| Challenge: | Spatial relation reasoning is a crucial task for multimodal large language models to understand the objective world. |
| Approach: | They propose a human-annotated spatial relation reasoning benchmark based on COCO2017 to improve MLLMs' spatial relation thinking. |
| Outcome: | The proposed benchmark achieves 48.14% accuracy, far below the human-level accuracy of 98.40%. |
GR1: Reinforcement-Enhanced LLM for Geoscience Reasoning (2026.findings-acl)
Copied to clipboard
Yule Xie, Jiaxin Ding, Cheng Deng, Shiqing Gao, Junran Zhang, Sibo Zhang, Zeyuan Wang, Ke Wu, Xin Ding, Luoyi Fu, Meng Jin, Xinbing Wang
| Challenge: | Recent advances in large language models have demonstrated RL's substantial capacity to enhance multi-step reasoning beyond what supervised instruction tuning achieves. |
| Approach: | They propose a framework that converts multimodal questions into descriptive text . they propose RL-enhanced geoscience reasoning that can be fine-tuned to a text-only level . |
| Outcome: | The proposed framework improves accuracy and accuracy on multimodal questions while preserving answerability and difficulty. |
TP-RAG: Benchmarking Retrieval-Augmented Large Language Model Agents for Spatiotemporal-Aware Travel Planning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on large language models (LLMs) focus on basic plan validity, but neglect critical aspects such as route efficiency, POI appeal, and real-time adaptability. |
| Approach: | They propose a benchmark for retrieval-augmented, spatiotemporal-aware travel planning that integrates retrieved trajectories with LLMs’ intrinsic reasoning. |
| Outcome: | The proposed framework improves spatial efficiency and POI rationality while challenging universality and robustness due to conflicting references and noisy data. |
SpatialWebAgent: Leveraging Large Language Models for Automated Spatial Information Extraction and Map Grounding (2025.acl-demo)
Copied to clipboard
| Challenge: | Understanding and extracting spatial information from text is vital for a wide range of applications, says nielsen . inherent complexity of geographic expressions in natural language presents significant hurdles for traditional extraction methods. |
| Approach: | They propose a system that leverages large language models to extract spatial information from natural language. |
| Outcome: | SpatialWebAgent is designed to extract, standardize, and ground spatial information from natural language text directly onto maps. |
SPARTQA: A Textual Question Answering Benchmark for Spatial Reasoning (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing studies have focused on the spatial reasoning capabilities of modern language models (LMs) however, there has been limited research into the spatial thinking capabilities of LMs. |
| Approach: | They propose a question-answering (QA) benchmark for spatial reasoning on natural language text which contains more realistic spatial phenomena not covered by prior work. |
| Outcome: | The proposed method significantly improves LMs' ability on spatial understanding, which in turn helps solve two external datasets, bAbI, and boolQ. |