| Challenge: | Visual7W has been widely used in assessing multiple-choice visual question-answering systems. |
| Approach: | They replicated a human experiment on Visual7W to examine the human-level performance of VQA. |
| Outcome: | The results show that the better a model performs on Visual7W, the better it aligns with human-level intelligence. |
Similar Papers
What’s Different between Visual Question Answering for Machine “Understanding” Versus for Accessibility? (2022.aacl-main)
Copied to clipboard
| Challenge: | Existing benchmarking datasets for visual question answering focus on machine "understanding" but it remains unclear how progress on those datasets corresponds to improvements in this real-world use case. |
| Approach: | They evaluate the visual question answering task by evaluating a variety of VQA models. |
| Outcome: | The proposed model can achieve high scores on tasks thought to require human-like comprehension, including image tagging and captioning. |
Question Modifiers in Visual Question Answering (2022.lrec-1)
Copied to clipboard
| Challenge: | Visual Question Answering (VQA) is a multi-disciplinary task that requires integration of several key disciplines. |
| Approach: | They develop a model that adds modifiers to questions based on object properties and spatial relationships using Amazon Mechanical Turk data. |
| Outcome: | The proposed model can improve when questions are modified to include more details. |
Uncovering the Full Potential of Visual Grounding Methods in VQA (2024.acl-long)
Copied to clipboard
| Challenge: | Visual Grounding (VG) methods in VQA aim to strengthen a model's reliance on question-relevant visual information. |
| Approach: | They propose to strengthen a model's reliance on question-relevant visual information by using a visual grounding method that is based on a question-related visual input. |
| Outcome: | The proposed methods can be much more effective when evaluation conditions are corrected. |
Delving Deeper into Cross-lingual Visual Question Answering (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing studies on cross-lingual VQA have reported poor zero-shot transfer performance of current multilingual multimodal Transformers . lack of multilingual resources has hindered development and evaluation of VQA methods beyond the English language . |
| Approach: | They analyze cross-lingual VQA across different question types of varying complexity . they show that simple modifications to the standard training setup can substantially reduce the transfer gap to monolingual English performance. |
| Outcome: | The proposed model significantly reduces the transfer gap to monolingual English performance . the proposed model also improves on question types and languages . |
A negative case analysis of visual grounding methods for VQA (2020.acl-main)
Copied to clipboard
| Challenge: | Existing Visual Question Answering (VQA) methods exploit dataset biases and spurious statistical correlations instead of producing correct answers for the right reasons. |
| Approach: | They propose to incorporate visual cues to better ground VQA models . they also propose a regularization effect which prevents over-fitting to linguistic priors . |
| Outcome: | The proposed method outperforms existing methods on the Visual Question Answering (VQA) dataset. |
Addressing Blind Guessing: Calibration of Selection Bias in Multiple-Choice Question Answering by Video Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing MCQA benchmarks fail to capture the full reasoning capabilities of video language models due to selection bias. |
| Approach: | They propose a method to reduce selection bias in video-to-text LLMs by suppressing "blind guessing" they propose 'bold' calibration technique to balance selection bias. |
| Outcome: | The proposed method reduces selection bias and improves model performance compared to existing methods. |
Benchmarking and Mitigating MCQA Selection Bias of Large Vision-Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing work has explored unimodal biases in visual question answering, but the problem of selection bias in Multiple-Choice Question Answering (MCQA) remains underexplored. |
| Approach: | They propose a method that mitigates bias without retraining and is compatible with frozen LVLMs. |
| Outcome: | The proposed method mitigates bias without retraining and is compatible with frozen LVLMs. |
Do explanations make VQA models more predictable to a human? (D18-1)
Copied to clipboard
| Challenge: | Existing explanations of a model's behavior are not used in interactive tasks like Visual Question Answering (VQA). |
| Approach: | They analyze existing explanations and their role in making a VQA model more predictable to a human by using human-in-the-loop approaches that treat the model as a black-box. |
| Outcome: | The proposed explanations make a model more predictable to humans, whereas human-in-the-loop approaches treat it as a black-box do. |
Fast or Slow? Integrating Fast Intuition and Deliberate Thinking for Enhancing Visual Question Answering (2025.acl-short)
Copied to clipboard
| Challenge: | Current approaches generate visual markers for all questions, generating excessive visual markers. |
| Approach: | They propose a plug-and-play approach that adapts to the complexity of questions . they propose combining fast intuitive judgments with deliberate analytical reasoning . |
| Outcome: | The proposed approach improves performance on four benchmarks on ScienceQA, TextQA, VizWiz, and MME. |
NegVQA: Can Vision Language Models Understand Negation? (2025.findings-acl)
Copied to clipboard
| Challenge: | NegVQA is a visual question answering (VQA) benchmark consisting of 7,379 two-choice questions covering diverse negation scenarios and image-question distributions. |
| Approach: | They propose a visual question answering benchmark consisting of 7,379 two-choice questions covering diverse negation scenarios and image-question distributions. |
| Outcome: | The proposed model fails to correctly interpret negation, leading to critical errors in interactive AI systems. |