Navigating the Nuances: A Fine-grained Evaluation of Vision-Language Navigation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | a new evaluation framework for vision-language navigation is proposed . current evaluation standards hinge on endpoint success rates and path alignment metrics . |
| Approach: | They propose a semi-automatic method for CFG construction with Large-Language Models . they induct data spanning five principal instruction categories and analyze them . |
| Outcome: | The proposed framework diagnoses current models for the Vision-Language Navigation task at a finer-grained level. |
Similar Papers
ULN: Towards Underspecified Vision-and-Language Navigation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing vision-and-language navigation models are brittle to multi-level language underspecification. |
| Approach: | They propose to use multi-level underspecified instructions to guide agents . they propose to learn GSS for navigation agent to ground multi- level instructions . experimental results show existing VLN models are still brittle to multi-language underspecification . |
| Outcome: | Experimental results show that the proposed framework outperforms baselines on ULN by 10% relative success rate across all levels. |
VLN-NF: Feasibility-Aware Vision-and-Language Navigation with False-Premise Instructions (2026.acl-long)
Copied to clipboard
| Challenge: | Existing Vision-and-Language Navigation benchmarks assume instructions are feasible and the referenced target exists. |
| Approach: | They propose a benchmark with false-premise instructions where the target is absent . they propose supervised room-level navigation with LLM/VLM-driven in-room exploration . |
| Outcome: | The proposed benchmark produces false-premise goals that are plausible but factually incorrect . ROAM achieves the best REV-SPL among compared methods, while baselines often under-explore and terminate prematurely under unreliable instructions. |
Sub-Instruction Aware Vision-and-Language Navigation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Despite significant advances, few previous works are able to fully utilize the strong correspondence between visual and textual sequences. |
| Approach: | They propose to provide agents with fine-grained annotations during training and provide them with sub-instructions and their corresponding paths. |
| Outcome: | The proposed method improves the performance of four state-of-the-art agents in a room-to-room (R2R) benchmark dataset. |
Diagnosing Vision-and-Language Navigation: What Really Matters (2022.naacl-main)
Copied to clipboard
Wanrong Zhu, Yuankai Qi, Pradyumna Narayana, Kazoo Sone, Sugato Basu, Xin Wang, Qi Wu, Miguel Eckstein, William Yang Wang
| Challenge: | Existing models claim to be able to align object tokens with specific visual targets, but there are non-negligible gaps between the two. |
| Approach: | They conduct diagnostic experiments to examine how the agents perceive multimodal input by ablation diagnostics input data. |
| Outcome: | The results show that indoor and outdoor navigation agents refer to object and direction tokens when making decisions. |
Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directions (2022.acl-long)
Copied to clipboard
| Challenge: | Vision-and-Language Navigation (VLN) is a research topic that is gaining attention in the field of artificial intelligence. |
| Approach: | They propose to build an embodied agent that can communicate with humans in natural language and navigate in real 3D environments. |
| Outcome: | This paper reviews current studies in the emerging field of vision-and-language navigation . it highlights limitations and opportunities for future work . |
Analyzing Generalization of Vision and Language Navigation to Unseen Outdoor Areas (2022.acl-long)
Copied to clipboard
| Challenge: | Recent work on visual-grounded navigation has focused on indoor scenarios with sharp drops in performance when testing on unseen data. |
| Approach: | They focus on visual agent navigation in outdoor scenarios with panorama images . they find that most gain in outdoor VLN on unseen data is due to specific features . |
| Outcome: | The results show a bias to specifics of graph representations of urban environments, demanding that VLN tasks grow in scale and diversity of geographical environments. |
Follow the Beaten Path: The Role of Route Patterns on Vision-Language Navigation Agents Generalization Abilities (2025.naacl-long)
Copied to clipboard
| Challenge: | Vision and language navigation (VLN) is a challenging task towards the creation of embodied agents. |
| Approach: | They propose a solution that combines visual and linguistic features to enable VLN . they propose augmentation of the training data to fill the gap in missing patterns . |
| Outcome: | The proposed solution fills the gap in missing patterns of training data. |
GROKE: Vision-Free Navigation Instruction Evaluation via Graph Reasoning on OpenStreetMap (2026.acl-long)
Copied to clipboard
| Challenge: | Existing evaluation metrics for navigation instructions are limited by licensing constraints and computational costs. |
| Approach: | They propose a vision-free training-free hierarchical LLM-based framework for evaluating navigation instructions using OpenStreetMap data. |
| Outcome: | The proposed framework reduces navigation error by 68.5% compared to baselines on the Map2Seq dataset. |
VLN-MME: Diagnosing MLLMs as Language-guided Visual Navigation Agents (2026.acl-long)
Copied to clipboard
| Challenge: | Multimodal Large Language Models have demonstrated remarkable capabilities across vision-language tasks, but their performance as embodied agents needs further exploration. |
| Approach: | They propose a framework to evaluate multimodal large language models as zero-shot agents . they find that enhancing prevalent agents with Chain-of-Thought reasoning and self-reflection leads to an unexpected performance decrease. |
| Outcome: | The proposed framework enables comparisons and component-level ablations across diverse MLLM architectures, agent designs, and navigation tasks. |
Are You Looking? Grounding to Multiple Modalities in Vision-and-Language Navigation (P19-1)
Copied to clipboard
| Challenge: | Existing models that ground language into visual appearance and route structure are outperforming their visual counterparts in unseen new environments. |
| Approach: | They propose to decompose the grounding procedure into a set of expert models with access to different modalities and ensemble them at prediction time. |
| Outcome: | The proposed model outperforms models with only route structure and visual features on the benchmark Room-to-Room dataset. |