PlanGPT-VL: Enhancing Urban Planning with Domain-Specific Vision-Language Models

Challenge:	Existing Vision-Language Models (VLMs) fail to analyze planning maps . specialized visual representations of land use zones, transportation networks, and development policies are needed to interpret complex planning maps.
Approach:	They propose a domain-specific VLM tailored for urban planning maps that employs three innovations: PlanAnno-V framework for high-quality VQA data synthesis, Critical Point Thinking (CPT) and PlanBench-V benchmark for systematic evaluation.
Outcome:	The new model outperforms general-purpose VLMs on planning map interpretation tasks.

Similar Papers

Overview

Challenge

Approach

Outcome

PlanGPT: Enhancing Urban Planning with a Tailored Agent Framework (2025.acl-industry)

Copied to clipboard

He Zhu, Guanhua Chen, Wenjia Zhang

Semantic Text Processing

Challenge:	Empirical tests demonstrate that PlanGPT framework has achieved advanced performance, providing comprehensive support that significantly enhances professional planning efficiency.
Approach:	They propose a specialized AI agent framework tailored for urban and spatial planning that integrates a customized local database retrieval system and domain-specific knowledge activation capabilities.
Outcome:	Empirical tests show that PlanGPT framework significantly improves planning efficiency . it integrates a customized database retrieval system, domain-specific knowledge activation capabilities, and advanced tool orchestration mechanisms.

On the Limit of Language Models as Planning Formalizers (2025.acl-long)

Copied to clipboard

Cassie Huang, Li Zhang

Language Models

Challenge:	Large Language Models can create plans that are neither executable nor verifiable in grounded environments.
Approach:	They use Large Language Models to generate a formal representation of the planning domain in some language, such as Planning Domain Definition Language (PDDL).
Outcome:	The proposed model outperforms the models directly generating plans while being robust to lexical perturbation.

TextLap: Customizing Language Models for Text-to-Layout Planning (2024.findings-emnlp)

Copied to clipboard

Jian Chen, Ruiyi Zhang, Yufan Zhou, Jennifer Healey, Jiuxiang Gu, Zhiqiang Xu, Changyou Chen

Language Models

Challenge:	Creating 2D graphical layouts from text alone is challenging in traditional settings.
Approach:	They propose to customize LLMs to allow users to generate professional looking layouts by simply inputting text instructions.
Outcome:	The proposed method outperforms existing benchmarks for document generation and graphical design benchmarks.

PlanningArena: A Modular Benchmark for Multidimensional Evaluation of Planning and Tool Learning (2025.acl-long)

Copied to clipboard

Zihan Zheng, Tianle Cui, Chuwen Xie, Jiahui Pan, Qianglong Chen, Lewei He

Reasoning

Challenge:	Recent studies have shown that LLMs can be significantly improved by integrating external tools.
Approach:	They propose a framework that integrates external tools into large language models to evaluate their ability to generate action plans.
Outcome:	The proposed framework evaluates the ability of large language models to generate action plans and generate action plan templates.

UrbanLLM: Autonomous Urban Activity Planning and Management with Large Language Models (2024.findings-emnlp)

Copied to clipboard

Yue Jiang, Qin Chao, Yile Chen, Xiucheng Li, Shuai Liu, Gao Cong

Language Models

Challenge:	UrbanLLM is a fine-tuned large language model designed to tackle diverse urban problems.
Approach:	They propose a fine-tuned large language model to tackle diverse urban problems . UrbanLLM decomposes urban-related queries into manageable sub-tasks .
Outcome:	The proposed model outperforms existing models in urban planning and management tasks.

Language Model as Planner and Formalizer under Constraints (2026.acl-long)

Copied to clipboard

Cassie Huang, Stuti Mohan, Ziyi Yang, Stefanie Tellex, Li Zhang

Language Models

Challenge:	Large language models (LLMs) have been widely used in planning but lack interpretability and control.
Approach:	They propose to augment widely used planning benchmarks with manually annotated, fine-grained, and rich natural language constraints spanning four formally defined categories.
Outcome:	The proposed model outperforms existing models in 4 state-of-the-art reasoning LLMs, 4 formal languages, and 4 datasets.

On the Fine-Grained Planning Abilities of VLM Web Agents (2025.findings-emnlp)

Copied to clipboard

Surgan Jandial, Yinong Oliver Wang, Andrea Bajcsy, Fernando De la Torre

Challenge:	Vision-Language Models (VLMs) have shown promise as web agents, yet their planning has been overlooked.
Approach:	They propose to examine VLMs’ ability to understand temporal relationships within web contexts and assess plans of actions across diverse scenarios.
Outcome:	The proposed models exhibit limited performance in the above skills and are not reliable to function as web agents.

Programming over Thinking: Efficient and Robust Multi-Constraint Planning (2026.acl-long)

Copied to clipboard

Derrick Goh Xin Deik, Quanyu Long, Zhengyuan Liu, Nancy F. Chen, Wenya Wang

Responsible & Trustworthy NLP

Challenge:	Existing large language model approaches lack flexibility in multi-constraint planning . SCOPE achieves state-of-the-art performance while lowering cost and latency .
Approach:	They propose a framework that disentangles query-specific problem reasoning from generic code execution.
Outcome:	The Scalable Code Planning Engine achieves state-of-the-art performance while lowering cost and latency.

Can LLMs Truly Plan? A Comprehensive Evaluation of Planning Capabilities (2025.findings-emnlp)

Copied to clipboard

Gayeon Jung, HyeonSeok Lim, Minjun Kim, Joon-ho Lim, KyungTae Lim, Hansaem Kim

Challenge:	Existing assessments of planning capabilities of large language models are limited to single-language or specific representation formats.
Approach:	a new benchmark is developed to assess the planning capabilities of large language models.
Outcome:	The Multi-Plan benchmark highlights performance disparities among models . language differences showed minimal impact, while mathematically structured representations improved accuracy .

ActPlan-1K: Benchmarking the Procedural Planning Ability of Visual Language Models in Household Activities (2024.emnlp-main)

Copied to clipboard

Ying Su, Zhan Ling, Haochen Shi, Cheng Jiayang, Yauwai Yim, Yangqiu Song

Multimodality

Challenge:	Large language models (LLMs) have been adopted to process textual task description and accomplish procedural planning in embodied AI tasks because of their powerful reasoning ability.
Approach:	They propose to evaluate the planning ability of large language models and multi-modal counterfactual vision language models (VLMs) using a multi-factual household activity simulator and a chatGPT task description to evaluate their reasoning ability.
Outcome:	The proposed benchmark evaluates the planning ability of multi-modal and counterfactual vision language models on a household activity simulator and a chatGPT task description.