II-MMR: Identifying and Improving Multi-modal Multi-hop Reasoning in Visual Question Answering (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing studies have focused on assessing the model’s overall accuracy without evaluating it on different reasoning cases. |
| Approach: | They propose a novel idea to identify and improve multi-modal multi-hop reasoning in VQA by using two new language prompts to find a reasoning path to reach its answer. |
| Outcome: | The proposed model improves multi-modal multi-hop reasoning in visual question answering (VQA) it finds that the proposed model is easy to answer, simply demanding “single-hop” reasoning, whereas only a few questions require “multi-hop.” |
Similar Papers
M3-VQA: A Benchmark for Multimodal, Multi-Entity, Multi-Hop Visual Question Answering (2026.acl-long)
Copied to clipboard
| Challenge: | Existing knowledge-based VQA benchmarks focus on coarse-grained categories and simple reasoning over single entities. |
| Approach: | They propose a knowledge-based Visual Question Answering benchmark to enhance multimodality evaluation. |
| Outcome: | The proposed benchmark improves evaluation of multimodal large language models in fine-grained multimodal entity understanding and complex multihop reasoning. |
MTabVQA: Evaluating Multi-Tabular Reasoning of Language Models in Visual Space (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing benchmarks address single tables or non-visual data, leaving a critical gap . MTabVQA comprises 3,745 complex question-answer pairs . |
| Approach: | They propose a benchmark specifically designed for multi-tabular visual question answering that measures the ability to parse diverse table images and correlate information across them. |
| Outcome: | The proposed benchmarks show that fine-tuning VLMs with MTabVQA-Instruct significantly improves their reasoning abilities. |
MM-Reasoner: A Multi-Modal Knowledge-Aware Framework for Knowledge-Based Visual Question Answering (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent knowledge-based visual question answering approaches miss visual information captured by captions and cannot fully utilize the visual information required to answer the question. |
| Approach: | They propose a framework that extracts visual information from an image and prompts an LLM to extract query-specific knowledge from the extracted textual information. |
| Outcome: | Empirical results show that MM-Reasoner achieves state-of-the-art performance on several KVQA datasets. |
MMCoQA: Conversational Question Answering over Text, Tables, and Images (2022.acl-long)
Copied to clipboard
| Challenge: | Existing conversational QA systems only use a single knowledge source, e.g., paragraphs or knowledge graph, and assume it contains enough evidence to extract answers to users' questions. |
| Approach: | They propose a task to answer users' questions with multimodal knowledge sources via multi-turn conversations using a multimodal dataset. |
| Outcome: | The proposed task brings a series of research challenges, including but not limited to priority, consistency, and complementarity of multimodal knowledge. |
Answering Questions by Meta-Reasoning over Multiple Chains of Thought (2023.emnlp-main)
Copied to clipboard
| Challenge: | Modern systems for multi-hop question answering (QA) break questions into a sequence of reasoning steps, termed chain-of-thought (CoT) Often, multiple chains are sampled and aggregated, but the intermediate steps themselves are discarded. |
| Approach: | They propose a method which prompts large language models to meta-reason over multiple chains of thought rather than aggregate their answers. |
| Outcome: | The proposed approach outperforms baselines on 7 multi-hop QA datasets. |
Learning to Ask: Multi-Decoder Fine-Tuning for Multi-Hop Visual Question Generation with External Knowledge (2026.findings-eacl)
Copied to clipboard
| Challenge: | Traditional supervised QG methods rely on tokenlevel alignment with fixed gold labels struggle to capture diverse valid question formulations. |
| Approach: | They propose a model-agnostic framework that integrates multimodal inputs with a multi-decoder architecture to optimize for multiple labels per sample. |
| Outcome: | The proposed framework improves fluency, reasoning depth, and relevance of visual questions. |
AIM-CoT: Active Information-driven Multimodal Chain-of-Thought for Vision-Language Reasoning (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for I-MCoT fail to capture dynamic needs of vision-language models . existing methods rely on attention signals, which are unreliable under severe granularity imbalance between brief textual query and informative image. |
| Approach: | They propose a framework that integrates specially selected visual evidence into the context of Vision-Language Models (VLMs) they propose 'AIM-CoT' to improve evidence selection and insertion triggering . |
| Outcome: | Experiments across three benchmarks and four backbones demonstrate the proposed framework’s consistent superiority. |
DRIVINGVQA: A Dataset for Interleaved Visual Chain-of-Thought in Real-World Driving Scenarios (2026.findings-eacl)
Copied to clipboard
| Challenge: | Chain-of-thought (CoT) prompting is a prompting strategy that improves reasoning in large language models, but its effectiveness in vision-language models remains limited due to over-reliance on textual cues and memorized knowledge. |
| Approach: | They propose a visual question-answering dataset derived from driving theory exams that incorporates textual explanations with visual tokens extracted from entities relevant to the reasoning process. |
| Outcome: | The proposed approach outperforms chain-of-thought prompting in large language models and vision-language models in real-world scenarios. |
Language Guided Visual Question Answering: Elevate Your Multimodal Language Model Using Knowledge-Enriched Prompts (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Visual question answering (VQA) is a task that requires an understanding of both the image and the question to provide a natural language answer. |
| Approach: | They propose a multimodal framework that leverages language guidance to answer questions more accurately. |
| Outcome: | The proposed framework improves on the multi-choice question-answering task using CLIP and BLIP models. |
Incorporating Probing Signals into Multimodal Machine Translation via Visual Question-Answering Pairs (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies show that multimodal machine translation systems exhibit decreased sensitivity to visual information when text inputs are complete. |
| Approach: | They propose to generate parallel VQA style pairs from source text to foster more robust cross-modal interaction. |
| Outcome: | The proposed approach generates parallel VQA style pairs from the source text, fostering more robust cross-modal interaction. |