Measuring and Improving Chain-of-Thought Reasoning in Vision-Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Vision-language models have demonstrated strong efficacy as visual assistants . however, evaluation of their reasoning capabilities requires a costly benchmark . |
| Approach: | They propose a pipeline to measure the reasoning consistency of vision-language models . they propose supervised fine-tuning of VLMs and feedback from LLMs . |
| Outcome: | The proposed framework reduces cost while ensuring the generation of a high-quality dataset. |
Similar Papers
Improve Vision Language Model Chain-of-thought Reasoning (2025.acl-long)
Copied to clipboard
Ruohong Zhang, Bowen Zhang, Yanghao Li, Haotian Zhang, Zhiqing Sun, Zhe Gan, Yinfei Yang, Ruoming Pang, Yiming Yang
| Challenge: | Current training recipes often rely on datasets dominated by short annotations with limited rationales, hindering the models' ability to generalize to tasks requiring comprehensive reasoning. |
| Approach: | They propose a two-stage post-training strategy that augments short answers with CoT reasoning generated by GPT-4o, enhancing the VLM's CoT capabilities through fine-tuning. |
| Outcome: | The proposed strategy enhances the model's CoT capabilities through fine-tuning and reinforcement learning. |
Render-of-Thought: Rendering Textual Chain-of-Thought as Images for Visual Latent Reasoning (2026.acl-long)
Copied to clipboard
| Challenge: | Recent work on Chain-of-Thought prompting imposes substantial computational overhead . lack of supervision obscures the analyzability of the latent reasoning chain. |
| Approach: | They propose a framework to render latent reasoning chain into images, making latent rationale explicit and traceable. |
| Outcome: | The proposed framework achieves 3-4 token compression and substantial inference acceleration compared to explicit CoT prompting. |
Journey Before Destination: On the importance of Visual Faithfulness in Slow Thinking (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing evaluations for visual hallucinations are narrow. |
| Approach: | They propose a framework that decomposes reasoning chains into perception versus reasoning steps and uses off-the-shelf VLM judges for step-level faithfulness. |
| Outcome: | The proposed framework reduces Unfaithful Perception Rate while preserving final-answer accuracy. |
A Closer Look at Bias and Chain-of-Thought Faithfulness of Large (Vision) Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Chain-of-thought reasoning improves performance of large language models, but is it faithfully reflecting internal processes? |
| Approach: | They propose a new evaluation pipeline for categorizing bias articulation patterns and a novel evaluation pipeline to examine CoT faithfulness in large vision-language models. |
| Outcome: | The proposed evaluation pipeline enables significantly more precise analysis of CoT reasoning than previous methods. |
Revealing the Seen, Imagining the Beyond: A Survey of Image-Grounded Chain-of-Thought Reasoning in Multimodal LLMs (2026.acl-long)
Copied to clipboard
| Challenge: | Recent advances in Multimodal Large Language Models (MLLMs) have shifted visual reasoning from tool-calling to end-to-end perceptionreasoning. |
| Approach: | They synthesize the emerging paradigm of Image-Grounded Chain-of-Thought (IG-CoT) they propose a method-centric taxonomy covering prompting, supervised fine-tuning, and reinforcement learning . |
| Outcome: | The proposed model is based on a method-centric taxonomy and benchmarks. |
Enhancing Advanced Visual Reasoning Ability of Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in Vision-Language (VL) research have sparked new benchmarks for complex visual reasoning, challenging models’ advanced reasoning ability. |
| Approach: | They propose a novel multi-modal in-context learning methodology to enhance LLMs’ contextual understanding and reasoning. |
| Outcome: | The proposed model achieves SOTA performance among all visual reasoning tasks and achieves a 'higher level of accuracy' than previous models. |
How Likely Do LLMs with CoT Mimic Human Reasoning? (2025.coling-main)
Copied to clipboard
| Challenge: | Using chain-of-thought to elicit reasoning capabilities is not always effective and accurate. |
| Approach: | They compare the reasoning process of LLMs with humans to understand the causal chain . they find that LLM deviates from the ideal causal chain, resulting in spurious correlations . |
| Outcome: | The proposed method does not improve performance or accurately represent reasoning processes in LLMs. |
V-ALPHASOCIAL: Benchmark and Self-Reflective Chain-of-Thought Generation for Visual Social Commonsense Reasoning (2025.findings-acl)
Copied to clipboard
Zongyu Lin, Zhikun Xu, Xiaohan Song, Yixin Wan, Xingcheng Yao, Tsung-Han Lin, Selina Song, Pranav Subbaraman, Ben Zhou, Kai-Wei Chang, Yizhou Sun
| Challenge: | Social commonsense reasoning is a multimodal task that requires both textual and visual cues. |
| Approach: | They propose a method that integrates visual cues into social commonsense reasoning tasks. |
| Outcome: | The proposed method improves social commonsense reasoning on a multimodal foundation model. |
DRIVINGVQA: A Dataset for Interleaved Visual Chain-of-Thought in Real-World Driving Scenarios (2026.findings-eacl)
Copied to clipboard
| Challenge: | Chain-of-thought (CoT) prompting is a prompting strategy that improves reasoning in large language models, but its effectiveness in vision-language models remains limited due to over-reliance on textual cues and memorized knowledge. |
| Approach: | They propose a visual question-answering dataset derived from driving theory exams that incorporates textual explanations with visual tokens extracted from entities relevant to the reasoning process. |
| Outcome: | The proposed approach outperforms chain-of-thought prompting in large language models and vision-language models in real-world scenarios. |
Does Chain-of-Thought Reasoning Help Mobile GUI Agents? An Empirical Study (2026.findings-acl)
Copied to clipboard
| Challenge: | Reasoning capabilities have improved vision-language models in domains like math, coding, and visual question-answering, but their impact on real-world applications remains unclear. |
| Approach: | They evaluate six pairs of VLMs by comparing their base and reasoning-enhanced versions across static and interactive benchmarks. |
| Outcome: | The reasoning-enhanced models perform better on static and interactive benchmarks than non-reasoning models. |