| Challenge: | Existing Visual Question Answering (VQA) methods exploit dataset biases and spurious statistical correlations instead of producing correct answers for the right reasons. |
| Approach: | They propose to incorporate visual cues to better ground VQA models . they also propose a regularization effect which prevents over-fitting to linguistic priors . |
| Outcome: | The proposed method outperforms existing methods on the Visual Question Answering (VQA) dataset. |
Similar Papers
Uncovering the Full Potential of Visual Grounding Methods in VQA (2024.acl-long)
Copied to clipboard
| Challenge: | Visual Grounding (VG) methods in VQA aim to strengthen a model's reliance on question-relevant visual information. |
| Approach: | They propose to strengthen a model's reliance on question-relevant visual information by using a visual grounding method that is based on a question-related visual input. |
| Outcome: | The proposed methods can be much more effective when evaluation conditions are corrected. |
WeaQA: Weak Supervision via Captions for Visual Question Answering (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for training visual question answering models rely on datasets with human-annotated image-quest-answer triplets. |
| Approach: | They propose a method to train models with synthetic Q-A pairs generated procedurally from captions. |
| Outcome: | The proposed method trains models with synthetic Q-A pairs generated from captions on three VQA benchmarks. |
‘Just because you are right, doesn’t mean I am wrong’: Overcoming a bottleneck in development and evaluation of Open-Ended VQA tasks (2021.eacl-main)
Copied to clipboard
Man Luo, Shailaja Keyur Sampat, Riley Tallman, Yankai Zeng, Manuha Vancha, Akarshan Sajja, Chitta Baral
| Challenge: | Existing visual question answering datasets assume only one ground truth answer for each question. |
| Approach: | They propose alternative answer sets (AAS) of ground-truth answers to address this limitation . they modify top VQA solvers to support multiple plausible answers for a question . |
| Outcome: | The proposed approach improves on the GQA dataset and shows that it is more efficient than previous approaches. |
A Multi-modal Debiasing Model with Dynamical Constraint for Robust Visual Question Answering (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have shown that many well-developed Visual Question Answering systems suffer from bias problem. |
| Approach: | They propose a way to mitigate bias problem by subtracting bias score from standard VQA base score. |
| Outcome: | The proposed method improves on the VQA v2.0 and VQA-CP V2,0 datasets. |
Measuring Faithful and Plausible Visual Grounding in VQA (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Lack of visual grounding (VG) in VQA systems can manifest in over-reliance on irrelevant image parts or a disregard for the visual modality entirely. |
| Approach: | They propose a new metric that measures if a model identifies question-relevant objects in the scene and relies on the information contained in the relevant objects when producing its answer. |
| Outcome: | The proposed metric measures if a model identifies question-relevant objects in the scene and relies on the information contained in the relevant objects when producing its answer. |
CommVQA: Situating Visual Question Answering in Communicative Contexts (2024.emnlp-main)
Copied to clipboard
| Challenge: | Current visual question answering models are trained on image-question pairs in isolation, but the questions people ask are dependent on their informational needs and prior knowledge about the image content. |
| Approach: | They propose a visual question-answer-as-question dataset that contains 1000 images and 8,949 question-announcer pairs to evaluate how situating images within naturalistic contexts shapes visual questions. |
| Outcome: | The proposed dataset contains 1000 images and 8,949 question-answer pairs. |
Multi-grained Attention with Object-level Grounding for Visual Question Answering (P19-1)
Copied to clipboard
| Challenge: | Current approaches to visual question answering train attention models from coarse-grained associations between sentences and images, which fail on small objects or uncommon concepts. |
| Approach: | They propose a multi-grained attention method that learns explicit word-object correspondence by word-level attention complementary to the sentence-image association. |
| Outcome: | The proposed method achieves competitive performance with state-of-the-art models on the VQA benchmark. |
NegVQA: Can Vision Language Models Understand Negation? (2025.findings-acl)
Copied to clipboard
| Challenge: | NegVQA is a visual question answering (VQA) benchmark consisting of 7,379 two-choice questions covering diverse negation scenarios and image-question distributions. |
| Approach: | They propose a visual question answering benchmark consisting of 7,379 two-choice questions covering diverse negation scenarios and image-question distributions. |
| Outcome: | The proposed model fails to correctly interpret negation, leading to critical errors in interactive AI systems. |
Causal Reasoning through Two Cognition Layers for Improving Generalization in Visual Question Answering (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing attempts to generalize VQA focus on unimodal aspects, overlooking enhancements in multimodal aspects. |
| Approach: | They propose to decompose the responsibility of each stage into distinct experts and a cognition-enabled component (CC) they prioritize answer predictions governed by pathways involving both CCs while disregarding answers produced by either CC. |
| Outcome: | The proposed model improves multimodal predictions by emphasizing causal reasoning factors. |
Overcoming Language Priors in Visual Question Answering via Distinguishing Superficially Similar Instances (2022.coling-1)
Copied to clipboard
| Challenge: | Existing VQA models rely on the superficial correlation between question type and frequent answers to make predictions, without really understanding the input. |
| Approach: | They propose a training framework that explicitly encourages the VQA model to distinguish between superficially similar instances. |
| Outcome: | The proposed framework achieves state-of-the-art performance on VQA-CP v2 . it explicitly encourages the model to distinguish between the superficially similar instances . |