BBA: Bi-Modal Behavioral Alignment for Reasoning with Large Vision-Language Models (2024.findings-acl)
Copied to clipboard
Xueliang Zhao, Xinting Huang, Tingchen Fu, Qintong Li, Shansan Gong, Lemao Liu, Wei Bi, Lingpeng Kong
| Challenge: | Multimodal reasoning is a key capability for large vision-language models . however, the vanilla Chain-of-Thought method fails to address critical steps in multi-step reasoning tasks. |
| Approach: | They propose a bi-modal Behavioral Alignment method to augment multimodal reasoning . they use domain-specific language to integrate multimodal information into a precise alternative form . |
| Outcome: | The proposed method significantly improves GPT-4V(ision) on geometry problem solving, chess positional advantage prediction and molecular property prediction. |
Similar Papers
Aligning Large and Small Language Models via Chain-of-Thought Reasoning (2024.eacl-long)
Copied to clipboard
| Challenge: | Chain-of-Thought (CoT) prompting empowers Large Language Models to solve complex reasoning tasks in a step-wise manner. |
| Approach: | They propose a method for aligning and transferring reasoning abilities between larger and smaller Language Models by using CoT-Demonstrations. |
| Outcome: | The proposed method outperforms baselines on question-answering and mathematical reasoning benchmarks. |
CoF-CoT: Enhancing Large Language Models with Coarse-to-Fine Chain-of-Thought Prompting for Multi-domain NLU Tasks (2023.emnlp-main)
Copied to clipboard
| Challenge: | Chain-of-Thought prompting is popular in reasoning tasks, but its application to Large Language Models (LLMs) in Natural Language Understanding (NLU) is under-explored. |
| Approach: | They propose a Coarse-to-Fine Chain-of-Thought approach that breaks down NLU tasks into multiple reasoning steps where LLMs can learn to acquire essential concepts. |
| Outcome: | The proposed approach is effective in assisting the LLMs adapt to multi-grained NLU tasks under zero-shot and few-shot multi-domain settings. |
Scaffolding Coordinates to Promote Vision-Language Coordination in Large Multi-Modal Models (2025.coling-main)
Copied to clipboard
| Challenge: | Existing prompting techniques for Large Multi-Modal Models (LMMs) focus on improving textual reasoning or leveraging tools for image preprocessing, lacking a simple and general visual prompting scheme to promote vision-language coordination. |
| Approach: | They propose a prompting scheme that scaffolds coordinates to promote vision-language coordination in Large Multi-Modal Models (LMMs) they overlay a dot matrix within the image as visual information anchors and leverage multi-dimensional coordinates as textual positional references. |
| Outcome: | Experiments on a wide range of vision-language tasks show the superiority of SCAFFOLD prompting over the textual Chain-of-Thought prompting. |
II-MMR: Identifying and Improving Multi-modal Multi-hop Reasoning in Visual Question Answering (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing studies have focused on assessing the model’s overall accuracy without evaluating it on different reasoning cases. |
| Approach: | They propose a novel idea to identify and improve multi-modal multi-hop reasoning in VQA by using two new language prompts to find a reasoning path to reach its answer. |
| Outcome: | The proposed model improves multi-modal multi-hop reasoning in visual question answering (VQA) it finds that the proposed model is easy to answer, simply demanding “single-hop” reasoning, whereas only a few questions require “multi-hop.” |
Beyond ’Aha!’: Toward Systematic Meta-Abilities Alignment in Large Reasoning Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Prior work has shown that outcome-based reinforcement learning (RL) can incidentally elicit advanced reasoning behaviors such as self-correction, backtracking, and verification. |
| Approach: | They explicitly align large reasoning models with three meta-abilities: deduction, induction, and abduction, using automatically generated, self-verifiable tasks. |
| Outcome: | The proposed model aligns models with deduction, induction, and abduction meta-abilities using automatically generated, self-verifiable tasks. |
RSVP: Reasoning Segmentation via Visual Prompting and Multi-modal Chain-of-Thought (2025.acl-long)
Copied to clipboard
Yi Lu, Jiawang Cao, Yongliang Wu, Bozheng Li, Licheng Tang, Yangguang Ji, Chong Wu, Jay Wu, Wenbo Zhu
| Challenge: | Recent advances in multi-modal learning have enhanced MLLMs' ability to reason about visual content. |
| Approach: | They propose a framework that unifies multi-step multimodal reasoning with grounded visual understanding. |
| Outcome: | The proposed framework surpasses state-of-the-art methods by +6.5 gIoU and +9.2 cIou on ReasonSeg and achieves 49.7 mAP on SegInW under zero-shot settings. |
VReST: Enhancing Reasoning in Large Vision-Language Models through Tree Search and Self-Reward Mechanism (2025.acl-long)
Copied to clipboard
| Challenge: | Large Vision-Language Models (LVLMs) have shown exceptional performance in multimodal tasks, but their effectiveness in complex visual reasoning is constrained. |
| Approach: | They propose a training-free approach that enhances Reasoning in Large Vision-Language Models . they propose integrating Monte Carlo Tree Search and Self-Reward mechanisms into the reasoning tree . |
| Outcome: | The proposed approach surpasses current prompting methods and secures state-of-the-art performance across three multimodal reasoning benchmarks. |
Re-Align: Aligning Vision Language Models via Retrieval-Augmented Direct Preference Optimization (2025.emnlp-main)
Copied to clipboard
Shuo Xing, Peiran Li, Yuping Wang, Ruizheng Bai, Yueqi Wang, Chan-Wei Hu, Chengxuan Qian, Huaxiu Yao, Zhengzhong Tu
| Challenge: | emergence of large Vision Language Models (VLMs) has broadened the capabilities of single-modal Large Language Model (LLM) but VLMs are prone to significant hallucinations, especially in the form of cross-modal inconsistencies. |
| Approach: | They propose a new alignment framework that leverages image retrieval to integrate both textual and visual preference signals. |
| Outcome: | The proposed framework mitigates hallucinations more effectively than previous methods . it maintains robustness and scalability across a wide range of VLM sizes and architectures . |
Enhancing Visual-Language Modality Alignment in Large Vision Language Models via Self-Improvement (2025.findings-naacl)
Copied to clipboard
Xiyao Wang, Jiuhai Chen, Zhaoyang Wang, Yuhang Zhou, Yiyang Zhou, Huaxiu Yao, Tianyi Zhou, Tom Goldstein, Parminder Bhatia, Taha Kass-Hout, Furong Huang, Cao Xiao
| Challenge: | Existing methods for visual and language alignment depend on external models or data, leading to uncontrollable and unstable results. |
| Approach: | They propose a framework that enhances visual and language alignment without external dependencies by incorporating an in-context self-critic mechanism that constructs preference pairs for tuning. |
| Outcome: | The proposed framework outperforms existing methods and improves performance on 14 hallucination and comprehensive benchmarks. |
DS-MHP: Improving Chain-of-Thought through Dynamic Subgraph-Guided Multi-Hop Path (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing knowledge graph methods lack adaptability in knowledge-intensive tasks with multiple entities and implicit multi-hop relations. |
| Approach: | They propose a zero-shot framework to enhance LLM reasoning in multi-entity relation tasks. |
| Outcome: | DS-MHP outperforms baselines and state-of-the-art methods on 12 datasets spanning commonsense, logical, symbolic, and arithmetic reasoning. |