Out of Sight, Not Out of Context? Egocentric Spatial Reasoning in VLMs Across Disjoint Frames (2025.emnlp-main)
Copied to clipboard
Sahithya Ravi, Gabriel Herbert Sarch, Vibhav Vineet, Andrew D Wilson, Balasaravanan Thoravi Kumaravel
| Challenge: | Disjoint-3DQA evaluates the spatial reasoning ability of embodied AI assistants based on egocentric video . it aims to catalyze future research at the intersection of vision, language, and embodie . |
| Approach: | They propose a generative QA benchmark that evaluates the ability of embodied AI assistants to integrate spatial cues across time by asking object pairs that are not co-visible in the same frame. |
| Outcome: | The proposed benchmark compares seven state-of-the-art VLMs and finds that they lag behind human performance by 28%, with steeper declines as the temporal gap widens. |
Similar Papers
AlanaVLM: A Multimodal Embodied AI Foundation Model for Egocentric Video Understanding (2024.findings-emnlp)
Copied to clipboard
Alessandro Suglia, Claudio Greco, Katie Baker, Jose Part, Ioannis Papaioannou, Arash Eshghi, Ioannis Konstas, Oliver Lemon
| Challenge: | Current Vision-Language Models (VLMs) focus on third-person view videos, neglecting the richness of egocentric perceptual experience. |
| Approach: | They propose to use the Egocentric Video Understanding Dataset (EVUD) to train VLMs on video captioning and question answering tasks specific to egocentric videos. |
| Outcome: | The proposed model outperforms open-source models including strong Socratic models using GPT-4 as a planner by 3.6% and outperformed Claude 3 and Gemini Pro Vision 1.0. |
Diagnosing Spatial Consistency across Perspectives and Viewpoints in Large Vision-Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing models assess spatial capabilities from a static, single-view and egocentric perspective, failing to capture the dynamic nature of real-world spatial cognition. |
| Approach: | They propose a benchmark to diagnose spatial reasoning capabilities using a 360 field of view. |
| Outcome: | The proposed benchmark evaluates allocentric and egocentric reasoning capabilities from multiple perspectives in high-quality 3D environments. |
EmbSpatial-Bench: Benchmarking Spatial Understanding for Embodied Tasks with Large Vision-Language Models (2024.acl-short)
Copied to clipboard
| Challenge: | Recent studies have revealed significant deficiencies of LVLMs in understanding visual contents, leaving the gap between current embodied intelligence and large vision-language models (LVLM) . |
| Approach: | They propose to use a benchmark to evaluate LVLMs' spatial understanding of embodied environments to evaluate their ability to understand visual contents. |
| Outcome: | The proposed benchmark is derived from embodied scenes and covers 6 spatial relationships from an egocentric perspective. |
Losing Visual Needles in Image Haystacks: Vision Language Models are Easily Distracted in Short and Long Contexts (2024.findings-emnlp)
Copied to clipboard
| Challenge: | evaluators of long-context vision language models (VLMs) have not kept up with the rapid development of open-weight long-constraint language models. |
| Approach: | They propose a dynamic benchmark generator for evaluating long-context reasoning in vision language models. |
| Outcome: | The proposed model can ignore irrelevant information when answering queries, showing that current models lack this capability. |
Do MLLMs Understand Pointing? Benchmarking and Enhancing Referential Reasoning in Egocentric Vision (2026.findings-acl)
Copied to clipboard
| Challenge: | Egocentric AI agents rely on pointing to resolve referential ambiguities in natural language commands. |
| Approach: | They propose a question-answering benchmark to evaluate and enhance pointing reasoning in egocentric views. |
| Outcome: | The proposed benchmark evaluates and enhances pointing reasoning in egocentric views. |
SpaRE: Enhancing Spatial Reasoning in Vision-Language Models with Synthetic Data (2025.acl-long)
Copied to clipboard
| Challenge: | Vision-language models struggle with spatial reasoning, a skill that humans excel at. |
| Approach: | They propose to use a spatial-reasoning Enhanced (SpaRE) VLM to improve spatial reasoning in visual question answering and robotics. |
| Outcome: | The proposed model achieves a 49% performance gain on the What's Up benchmark while maintaining strong results on general tasks. |
Embodied-Reasoner: Synergizing Visual Search, Reasoning, and Action for Embodied Interactive Tasks (2026.acl-long)
Copied to clipboard
Wenqi Zhang, Mengna Wang, Gangao Liu, Huixin Xu, Yiwei Jiang, Yongliang Shen, Guiyang Hou, Zhe Zheng, Hang Zhang, Xin Li, Jiajun Liu, Weiming Lu, Peng Li, Yueting Zhuang
| Challenge: | Recent advances in reasoning models have demonstrated remarkable capabilities on mathematical and coding tasks, but their effectiveness in embodied domains remains largely unexplored. |
| Approach: | They propose a reasoning model for interactive embodied tasks that synthesizes 9.3k coherent Observation-Thought-Action trajectories containing 64k ego-centric images and 90k diverse reasoning processes. |
| Outcome: | The proposed model outperforms existing visual reasoning models by +9%, 24%, and +13% on long-horizon tasks. |
CityCube: Benchmarking Cross-view Spatial Reasoning on Vision-Language Models in Urban Environments (2026.acl-long)
Copied to clipboard
Haotian Xu, Yue Hu, Zhengqiu Zhu, Chen Gao, Ziyou Wang, Junreng Rao, Wenhao Lu, Weishi Li, Quanjun Yin, Yong Li
| Challenge: | Existing benchmarks focus on indoor or street settings, overlooking challenges of open-ended urban spaces. |
| Approach: | They propose a benchmark to probe cross-view spatial reasoning capabilities of current VLMs in urban settings. |
| Outcome: | The citycube benchmark examines the performance of current vision-language models in urban environments. |
CoV: Chain-of-View Prompting for Spatial Reasoning (2026.findings-acl)
Copied to clipboard
Haoyu Zhao, Akide Liu, Zeyu Zhang, Weijie Wang, Feng Chen, Ruihan Zhu, Gholamreza Haffari, Bohan Zhuang
| Challenge: | Embodied question answering requires collecting context that is distributed across multiple viewpoints . most recent vision–language models (VLMs) are constrained to a fixed and finite set of input views . |
| Approach: | They propose a training-free, test-time reasoning framework that transforms a VLM into an active viewpoint reasoner through a coarse-to-fine exploration process. |
| Outcome: | The proposed framework improves LLM-Match performance by 11.98% on four mainstream VLMs. |
VideoQA-TA: Temporal-Aware Multi-Modal Video Question Answering (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods for video question answering align visual or textual features directly with large language models, limiting the deep semantic association between modalities and hindering a comprehensive understanding of interactions within spatial and temporal contexts. |
| Approach: | They propose a temporal-aware framework for multi-modal video question answering that aligns videos and questions at fine-grained levels. |
| Outcome: | The proposed framework improves reasoning ability and accuracy of videoQA by aligning videos and questions at fine-grained levels. |