Progressive Multimodal Search and Reasoning for Knowledge-Intensive Visual Question Answering (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to knowledge-intensive visual question answering lack mechanisms to revise misdirected reasoning. |
| Approach: | They propose a framework that progressively constructs a structured reasoning trajectory . they use dual-scope queries to retrieve diverse knowledge from heterogeneous knowledge bases . |
| Outcome: | The proposed framework improves retrieval recall and end-to-end answer accuracy. |
Similar Papers
Seeing Beyond: Enhancing Visual Question Answering with Multi-Modal Retrieval (2025.coling-industry)
Copied to clipboard
| Challenge: | Multi-modal Large language models still suffer from model hallucination and lack of specific knowledge when answering challenging questions. |
| Approach: | They propose to use a multi-modal retrieval augmented generation method to integrate knowledge from all modalities into a model to enable alignment between query and knowledge. |
| Outcome: | The proposed method achieves significant performance improvement on the VQA dataset. |
MAGIC-VQA: Multimodal And Grounded Inference with Commonsense Knowledge for Visual Question Answering (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing Large Vision-Language Models (LVLMs) lack integrated commonsense knowledge . lack of integrated common knowledge limits their robustness and accuracy in VQA . |
| Approach: | They propose a framework to enhance multimodal inference by integrating commonsense reasoning. |
| Outcome: | MAGIC-VQA improves comprehensive benchmark datasets, surpassing existing models in tasks requiring advanced commonsense reasoning. |
OMGM: Orchestrate Multiple Granularities and Modalities for Efficient Multimodal Retrieval (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for Knowledge-Based Visual Question Answering lack multimodal retrieval . large language models (LLMs) have demonstrated remarkable generalization and reasoning capabilities in text-based systems. |
| Approach: | They propose a multimodal vision-language retrieval-augmented generation system that harmonizes multiple modalities and modality to enhance retrieval. |
| Outcome: | The proposed system achieves state-of-the-art retrieval performance and competitive answers on InfoSeek and Encyclopedic-VQA benchmarks. |
Retrieval Augmented Visual Question Answering with Outside Knowledge (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent systems that use Dense Passage Retrieval (DPR) for visual question answering use Denses trained separately from answer generation. |
| Approach: | They propose a joint training scheme that integrates DPR with answer generation to improve the performance of OK-VQA systems. |
| Outcome: | The proposed model outperforms existing systems with strong Dense Passage Retrieval (DPR) it yields significant improvements in answer quality and computation required for training. |
Retrieving Multimodal Prompts for Generative Visual Question Answering (2023.findings-acl)
Copied to clipboard
| Challenge: | Visual question answering (VQA) is a multimodal machine learning problem that challenges a model to answer a question posed about an image. |
| Approach: | They propose a generative model enhanced by multimodal prompt retrieval that integrates retrieved prompts and multimodal features to generate answers in free text. |
| Outcome: | The proposed model outperforms its non-retrieval counterpart by 30% on medical VQA tasks. |
Fast or Slow? Integrating Fast Intuition and Deliberate Thinking for Enhancing Visual Question Answering (2025.acl-short)
Copied to clipboard
| Challenge: | Current approaches generate visual markers for all questions, generating excessive visual markers. |
| Approach: | They propose a plug-and-play approach that adapts to the complexity of questions . they propose combining fast intuitive judgments with deliberate analytical reasoning . |
| Outcome: | The proposed approach improves performance on four benchmarks on ScienceQA, TextQA, VizWiz, and MME. |
WikiSeeker: Rethinking the Role of Vision-Language Models in Knowledge-Based Visual Question Answering (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for Knowledge-Based Visual Question Answering rely on images as the retrieval key, and often overlook or misplace the role of Vision-Language Models (VLMs) |
| Approach: | They propose a multi-modal RAG framework that assigns VLMs two specialized agents: a Refiner and an Inspector. |
| Outcome: | Experiments on EVQA, InfoSeek, and M2KR show that the proposed framework achieves state-of-the-art performance with significant improvements in both retrieval accuracy and answer quality. |
EchoSight: Advancing Visual-Language Models with Wiki Knowledge (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing knowledge-based visual question answering systems struggle with these tasks due to limited integration of external knowledge. |
| Approach: | They propose a framework that enables large language models to answer visual questions requiring encyclopedic knowledge. |
| Outcome: | The proposed framework improves retrieval outcomes and accuracy of knowledge-based visual question answering tasks. |
Zero-shot Multimodal Document Retrieval via Cross-modal Question Generation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing multimodal large language models struggle when faced with unseen domains or languages. |
| Approach: | They propose a framework that leverages the broad knowledge of an MLLM to generate cross-modal pre-questions (preQs) before retrieval. |
| Outcome: | Experiments show that PREMIR outperforms existing retrievers on out-of-distribution benchmarks, including closed-domain and multilingual settings, outperforming strong baselines across all metrics. |
M3-VQA: A Benchmark for Multimodal, Multi-Entity, Multi-Hop Visual Question Answering (2026.acl-long)
Copied to clipboard
| Challenge: | Existing knowledge-based VQA benchmarks focus on coarse-grained categories and simple reasoning over single entities. |
| Approach: | They propose a knowledge-based Visual Question Answering benchmark to enhance multimodality evaluation. |
| Outcome: | The proposed benchmark improves evaluation of multimodal large language models in fine-grained multimodal entity understanding and complex multihop reasoning. |