RUN through the Streets: A New Dataset and Baseline Models for Realistic Urban Navigation (D19-1)
Copied to clipboard
| Challenge: | Existing work on map-based NL navigation relies on small artificial worlds with a fixed set of entities known in advance. |
| Approach: | They propose a task to interpret navigation instructions in natural language (NL) they use a dataset aligned with real, dense, urban maps to study neural architectures . |
| Outcome: | The proposed task is based on a dataset of 2515 navigation instructions aligned with real routes over three regions of Manhattan. |
Similar Papers
Generating Landmark Navigation Instructions from Maps as a Graph-to-Text Problem (2021.acl-long)
Copied to clipboard
| Challenge: | Current navigation services based on turns and distances of named streets . humans use efficient mode of navigation based around visible and salient landmarks . |
| Approach: | They propose a neural model that takes OpenStreetMap representations as input and learns to generate navigation instructions that contain salient landmarks from human natural language instructions. |
| Outcome: | The proposed model can generate navigation instructions that contain salient landmarks from openStreetMap . it is based on a dataset of 7,672 instances verified by human navigation in Street View . |
Can MLLMs Find Their Way in a City? Exploring Emergent Navigation from Web-Scale Knowledge (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing evaluation benchmarks for multimodal large language models (MLLMs) are language-centric or heavily reliant on simulated environments, rarely probing the nuanced, knowledge-intensive reasoning essential for practical, real-world scenarios. |
| Approach: | They propose a task of Sparsely Grounded Visual Navigation to evaluate MLLM-driven agents in city navigation in four diverse global cities. |
| Outcome: | The proposed benchmark encompassing four diverse global cities evaluates agents' decision-making abilities in city navigation. |
LLM-driven Instruction Following: Progresses and Concerns (2023.emnlp-tutorial)
Copied to clipboard
| Challenge: | a tutorial on task instruction is aimed at researchers and practitioners interested in NLP generalization . labeled examples are unlikely to be available in large numbers or do not exist . |
| Approach: | This tutorial will examine the progress of natural language processing (NLP) using labeled examples. authors propose that task instructions act as a novel resource for supervision. |
| Outcome: | This tutorial aims to answer questions about instruction-driven NLP . it focuses on the use of task instructions in a low-shot scenario . |
Recognition of Implicit Geographic Movement in Text (2020.lrec-1)
Copied to clipboard
| Challenge: | a growing field of research is analyzing the geographic movement of humans, animals, and other entities. |
| Approach: | They created a corpus of sentences labeled as describing geographic movement or not . they used hand labeling, crowd voting and machine learning to predict more labels . |
| Outcome: | a new method uses hand labeling, crowd voting and machine learning to predict more labels. |
Translating Navigation Instructions in Natural Language to a High-Level Plan for Behavioral Robot Navigation (D18-1)
Copied to clipboard
Xiaoxue Zang, Ashwini Pokle, Marynel Vázquez, Kevin Chen, Juan Carlos Niebles, Alvaro Soto, Silvio Savarese
| Challenge: | Existing models for translating free-form natural language instructions to a high-level plan for behavioral robot navigation are difficult due to the variability in the way people describe routes. |
| Approach: | They propose an end-to-end deep learning model for translating free-form natural language instructions to a high-level plan for robot navigation. |
| Outcome: | The proposed model significantly outperforms baseline approaches on a new dataset containing 10,050 pairs of navigation instructions. |
UrbanLLM: Autonomous Urban Activity Planning and Management with Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | UrbanLLM is a fine-tuned large language model designed to tackle diverse urban problems. |
| Approach: | They propose a fine-tuned large language model to tackle diverse urban problems . UrbanLLM decomposes urban-related queries into manageable sub-tasks . |
| Outcome: | The proposed model outperforms existing models in urban planning and management tasks. |
Can LLMs Learn to Map the World from Local Descriptions? (2026.acl-long)
Copied to clipboard
| Challenge: | Recent advances in large language models have demonstrated strong capabilities in tasks such as code generation and mathematical reasoning. |
| Approach: | They investigate whether large language models can construct coherent global spatial cognition by integrating fragmented relational descriptions. |
| Outcome: | The proposed models can generalize to unseen spatial relationships and exhibit latent representations aligned with real-world spatial distributions. |
Follow the Beaten Path: The Role of Route Patterns on Vision-Language Navigation Agents Generalization Abilities (2025.naacl-long)
Copied to clipboard
| Challenge: | Vision and language navigation (VLN) is a challenging task towards the creation of embodied agents. |
| Approach: | They propose a solution that combines visual and linguistic features to enable VLN . they propose augmentation of the training data to fill the gap in missing patterns . |
| Outcome: | The proposed solution fills the gap in missing patterns of training data. |
ALFRED-L: Investigating the Role of Language for Action Learning in Interactive Visual Environments (2022.emnlp-main)
Copied to clipboard
Arjun Akula, Spandana Gella, Aishwarya Padmakumar, Mahdi Namazifar, Mohit Bansal, Jesse Thomason, Dilek Hakkani-Tur
| Challenge: | Embodied Vision and Language Task Completion requires an embodied agent to interpret natural language instructions and egocentric visual observations to navigate through and interact with environments. |
| Approach: | They propose to use ALFRED to test whether models can adapt to tasks not seen during training that require the same types of language understanding as ALFred. |
| Outcome: | The proposed model can adapt to tasks that require the same types of language understanding as ALFRED-L. |
NavA3: Understanding Any Instruction, Navigating Anywhere, Finding Anything (2026.acl-long)
Copied to clipboard
Lingfeng Zhang, Xiaoshuai Hao, Yingbo Tang, Haoxiang Fu, Xinyu Zheng, Pengwei Wang, Zhongyuan Wang, Wenbo Ding, Shanghang Zhang
| Challenge: | Existing embodied navigation methods struggle with such tasks due to their limitations in comprehending high-level human instructions and localizing objects with an open vocabulary. |
| Approach: | They propose a hierarchical framework for long-horizon navigation that integrates human instructions with 3D scene views. |
| Outcome: | The proposed model achieves SOTA results and can complete long-horizon navigation tasks across different robot embodiments in real-world environments. |