Unleashing Spatial Reasoning in Multimodal Large Language Models via Textual Representation Guided Reasoning (2026.acl-long)
Copied to clipboard
| Challenge: | Existing Multimodal Large Language Models struggle with 3D spatial reasoning as they fail to construct structured abstractions of the 3D environment depicted in video inputs. |
| Approach: | They propose a prompting method that induces MLLMs to generate 3D representations as reasoning traces for more accurate spatial question answering. |
| Outcome: | Extensive experiments on VSI-Bench and OST-Bech show that TRACE improves over prior prompting strategies across a diverse range of MLLM backbones. |
Similar Papers
Can Multimodal Large Language Models Understand Spatial Relations? (2025.acl-long)
Copied to clipboard
| Challenge: | Spatial relation reasoning is a crucial task for multimodal large language models to understand the objective world. |
| Approach: | They propose a human-annotated spatial relation reasoning benchmark based on COCO2017 to improve MLLMs' spatial relation thinking. |
| Outcome: | The proposed benchmark achieves 48.14% accuracy, far below the human-level accuracy of 98.40%. |
Multimodal Causal Reasoning Benchmark: Challenging Multimodal Large Language Models to Discern Causal Links Across Modalities (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing MLLMs lack robustness in multimodal causal reasoning compared to their performance in textual settings. |
| Approach: | They propose a novel multimodal chain-of-thought (CoT) reasoning benchmark that leverages siamese images and text pairs to challenge MLLMs. |
| Outcome: | The proposed benchmark leverages siamese images and text pairs to challenge MLLMs. |
Position: Multimodal Large Language Models Can Significantly Advance Scientific Reasoning (2026.findings-acl)
Copied to clipboard
Yibo Yan, Shen Wang, Jiahao Huo, Jingheng Ye, Zhendong Chu, Xuming Hu, Philip S. Yu, Carla P Gomes, Bart Selman, Qingsong Wen
| Challenge: | Current scientific reasoning models struggle with generalization across domains and fall short of multimodal perception. |
| Approach: | They propose to use multimodal large language models to integrate text, images, and other modalities to enhance scientific reasoning. |
| Outcome: | The proposed models can integrate text, images, and other modalities and improve reasoning across disciplines. |
Look and Think: Efficient Multimodal Reasoning via Modality-Decoupled Compression (2026.findings-acl)
Copied to clipboard
| Challenge: | Multimodal large language models have strong performance on visual question answering benchmarks . however, their inference efficiency is severely constrained by the rapidly growing context . |
| Approach: | They propose a modality-decoupled compression method that enables efficient multimodal inference . they propose to evict visual tokens whenever visual grounding is unnecessary . |
| Outcome: | The proposed method reduces the average context length by up to 57% while maintaining comparable performance to the standard MLLM baseline. |
VQAGuider: Guiding Multimodal Large Language Models to Answer Complex Video Questions (2025.acl-long)
Copied to clipboard
| Challenge: | Multimodal large language models (MLLMs) can grasp the intention of a question and decomposing it to a series of visual recognition sub-tasks to find out the answer with the help of an agent. |
| Approach: | They propose a framework for multimodal large language models to grasp the intention of a question and decompose it into a series of visual recognition sub-tasks to find out the answer. |
| Outcome: | The proposed framework improves the accuracy of complex video-related questions by 29.6% and 17.2% on CVQA and the existing VQA datasets. |
An Empirical Analysis on Spatial Reasoning Capabilities of Large Multimodal Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large Multimodal Models (LMMs) have shown impressive generalization ability on vision and language tasks, but their spatial understanding is under-explored. |
| Approach: | They construct a VQA dataset to analyze LMMs' spatial reasoning capabilities. |
| Outcome: | The proposed model is stronger at basic object detection than complex spatial reasoning. |
Probing Multimodal Large Language Models for Global and Local Semantic Representations (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing studies have focused on the ability of MLLMs to generate single tokens one by one, while lacking studies about how their representation vectors can encode global multimodal information. |
| Approach: | They propose to use image-caption corpus to train Multimodal Large Language Models (MLLMs) . they find that the topmost layers encode more global semantic information . |
| Outcome: | The proposed models can encode more global semantic information, rather than the topmost layers, and perform better on visual-language entailment tasks. |
From Multimodal LLM to Human-level AI: Modality, Instruction, Reasoning, Efficiency and beyond (2024.lrec-tutorials)
Copied to clipboard
| Challenge: | This tutorial aims to deliver a comprehensive review of cutting-edge research in MLLMs. |
| Approach: | This tutorial will review cutting-edge research in MLLMs and examine the impact of ML in learning and reasoning. |
| Outcome: | This course will review cutting-edge research in MLLMs and examine the impact of ML models on learning, learning, and multimodal reasoning. |
Probing Logical Reasoning of MLLMs in Scientific Diagrams (2025.emnlp-main)
Copied to clipboard
| Challenge: | logical reasoning is key to real-world applications like science education, environmental monitoring, and medical diagnostics. |
| Approach: | They construct visual questions that follow seven structured templates with progressively more complex reasoning involved. |
| Outcome: | The proposed models perform logical inferences based on visual information. |
Forgotten Polygons: Multimodal Large Language Models are Shape-Blind (2025.findings-acl)
Copied to clipboard
William Rudman, Michal Golovanevsky, Amir Bar, Vedant Palit, Yann LeCun, Carsten Eickhoff, Ritambhara Singh
| Challenge: | Multimodal Large Language Models struggle with visual reasoning, despite strong performance on vision-language tasks. |
| Approach: | They propose a visually cued chain-of-thought prompting that enhances multi-step mathematical reasoning by explicitly referencing visual annotations in diagrams. |
| Outcome: | The proposed model improves GPT-4o's accuracy on an irregular polygon side-counting task from 7% to 93%. |