Exploring Spatial Schema Intuitions in Large Language and Vision Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models excel in varied NLP tasks, but lack a direct connection between sensory perception and physical action. |
| Approach: | They examine whether large language models capture implicit human intuitions about building blocks of language . they employ spatial cognitive foundations developed through early sensorimotor experiences . |
| Outcome: | The proposed model captures implicit human intuitions about building blocks of language without a tangible connection to embodied experiences. |
Similar Papers
Can LLMs Learn to Map the World from Local Descriptions? (2026.acl-long)
Copied to clipboard
| Challenge: | Recent advances in large language models have demonstrated strong capabilities in tasks such as code generation and mathematical reasoning. |
| Approach: | They investigate whether large language models can construct coherent global spatial cognition by integrating fragmented relational descriptions. |
| Outcome: | The proposed models can generalize to unseen spatial relationships and exhibit latent representations aligned with real-world spatial distributions. |
Does Visual Grounding Enhance the Understanding of Embodied Knowledge in Large Language Models? (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Despite significant progress in multimodal language models, it remains unclear whether visual grounding enhances their understanding of embodied knowledge compared to text-only models. |
| Approach: | They propose to assess vision-language models’ perceptual abilities across different sensory modalities through vector comparison and question-answering tasks with over 1,700 questions. |
| Outcome: | The proposed benchmark assesses the models’ perceptual abilities across different sensory modalities through vector comparison and question-answering tasks with over 1,700 questions. |
UrbanVideo-Bench: Benchmarking Vision-Language Models on Embodied Intelligence with Video Data in Urban Spaces (2025.acl-long)
Copied to clipboard
Baining Zhao, Jianjie Fang, Zichao Dai, Ziyou Wang, Jirong Zha, Weichen Zhang, Chen Gao, Yue Wang, Jinqiang Cui, Xinlei Chen, Yong Li
| Challenge: | Large multimodal models exhibit remarkable intelligence, yet their embodied cognitive abilities during motion in open-ended urban aerial spaces remain to be explored. |
| Approach: | They propose a benchmark to evaluate whether large multimodal models can process continuous first-person visual observations like humans. |
| Outcome: | The proposed model can process first-person visual observations like humans, enabling recall, perception, reasoning, and navigation. |
Can Language Models Understand Physical Concepts? (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing language models do not understand basic physical concepts in the human world. |
| Approach: | They propose a method to transfer embodied knowledge from visual models to LMs . they use visual concepts and embodies concepts learned from interaction with the world . |
| Outcome: | The proposed method achieves comparable performance with scaling up parameters of LMs 134. |
POSQA: Probe the World Models of LLMs with Size Comparisons (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Embodied language comprehension emphasizes that language understanding is not only mental processing in the brain but also involves interactions with the physical and social environment. |
| Approach: | They propose to use a physical object size question to examine the extremity of large language models to test their embodied comprehension. |
| Outcome: | The proposed dataset shows that even the largest LLMs perform poorly under the zero-shot setting. |
Do LLMs Capture Embodied Cognition and Cultural Variation? Cross-Linguistic Evidence from Demonstratives (2026.acl-long)
Copied to clipboard
| Challenge: | a new study examines whether large language models acquire embodied cognition and cultural conventions from training data . demonstratives are a natural lens for evaluating linguistic phenomena that reflect cultural variation . aaron e. duan and j. nà: "the complexity of the language model is a major challenge for LLMs" |
| Approach: | They introduce demonstratives as a probe for grounded knowledge by analyzing 6,400 responses from 320 native speakers. |
| Outcome: | The proposed model fails to understand proximal–distal contrast and shows no cultural differences . the proposed model is a new probe for evaluating embodied cognition and cultural conventions . |
Systematic Analysis of Image Schemas in Natural Language through Explainable Multilingual Neural Language Processing (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods for automatic detection of image schemas in natural language rely on specific assumptions about word classes as indicators of spatio-temporal events. |
| Approach: | They propose to train a supervised classifier that classifies natural language expressions into image schemas using a large dataset of examples from image schema literature. |
| Outcome: | The proposed model performs best in German, Russian, and French, and is based on a small dataset of examples from image schema literature. |
Large Language Models: The Need for Nuance in Current Debates and a Pragmatic Perspective on Understanding (2023.emnlp-main)
Copied to clipboard
| Challenge: | Current Large Language Models (LLMs) are unparalleled in their ability to generate grammatically correct, fluent text. |
| Approach: | They argue that LLMs only parrot statistical patterns in training data and that language learning in LLM cannot inform human language learning. |
| Outcome: | The proposed model can generate grammatically correct, fluent text without requiring human intervention. |
Can LLMs See Without Pixels? Benchmarking Spatial Intelligence from Textual Descriptions (2026.findings-acl)
Copied to clipboard
Zhongbin Guo, Zhen Yang, Yushan Li, Xinyue Zhang, Wenyu Gao, Jiacheng Wang, Chengzhi Li, Xiangrui Liu, Ping Jian
| Challenge: | Existing advances in Spatial Intelligence rely on vision-Language Models . however, a critical question remains: does spatial understanding originate from visual encoders? |
| Approach: | They propose to evaluate the SI performance of Large Language Models without pixel-level input. |
| Outcome: | The proposed benchmark challenges large language models to perform symbolic reasoning rather than visual pattern matching. |
LLMs as a synthesis between symbolic and distributed approaches to language (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a fierce battle is being fought between symbolic and distributed approaches to language and cognition . a recent study shows that morphosyntactic knowledge is encoded in a near-discrete fashion in LLMs . |
| Approach: | a new position paper examines the role of distributed and distributed approaches in language learning . authors argue that deep learning models represent a synthesis between the two traditions . |
| Outcome: | a new position paper shows that deep learning models for language represent a synthesis between the two traditions. |