| Challenge: | EASE is a diagnostic tool for Visual Question Answering (VQA) it quantifies the difficulty of an image, question sample. |
| Approach: | They propose a diagnostic tool which quantifies the difficulty of an image, question sample. |
| Outcome: | The proposed tool can be used to select the most-informative samples for training/fine-tuning. |
Similar Papers
Tell-and-Answer: Towards Explainable Visual Question Answering using Attributes and Captions (D18-1)
Copied to clipboard
| Challenge: | Existing approaches to visual question answering represent images using pre-trained CNNs . but they rarely provide any insight, apart from the answer, into the VQA process . |
| Approach: | They propose to break up the end-to-end VQA into two steps: explaining and reasoning . they first extract attributes and generate descriptions as explanations for an image . a reasoning module utilizes these explanations in place of the image to infer an answer . |
| Outcome: | The proposed system achieves comparable performance with baselines, but with added benefits of explanability and the ability to improve with higher quality explanations. |
What’s Different between Visual Question Answering for Machine “Understanding” Versus for Accessibility? (2022.aacl-main)
Copied to clipboard
| Challenge: | Existing benchmarking datasets for visual question answering focus on machine "understanding" but it remains unclear how progress on those datasets corresponds to improvements in this real-world use case. |
| Approach: | They evaluate the visual question answering task by evaluating a variety of VQA models. |
| Outcome: | The proposed model can achieve high scores on tasks thought to require human-like comprehension, including image tagging and captioning. |
Beyond Accuracy: A Consolidated Tool for Visual Question Answering Benchmarking (2021.emnlp-demo)
Copied to clipboard
| Challenge: | Existing evaluation tools for general Visual Question Answering (VQA) systems are limited to answering accuracy, but they can be used to evaluate performance in real-world scenarios. |
| Approach: | They propose a browser-based benchmarking tool with an API for easy integration of new models and datasets to keep up with the fast-changing landscape of VQA. |
| Outcome: | The proposed tool tests generalization capabilities of models across multiple datasets and includes metrics that measure biases and uncertainty to further explain model behavior. |
ConceptBert: Concept-Aware Representation for Visual Question Answering (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Visual Question Answering (VQA) is a challenging task that has received increasing attention from both the computer vision and the natural language processing communities. |
| Approach: | They propose an algorithm which learns a joint Concept-Vision-Language embedding for questions which require common sense knowledge from external structured content. |
| Outcome: | The proposed model is based on the Outer Knowledge-VQA and VQA datasets. |
All You May Need for VQA are Image Captions (2022.naacl-main)
Copied to clipboard
| Challenge: | Visual Question Answering (VQA) has benefited from increasingly sophisticated models, but has not enjoyed the same level of engagement in terms of data creation. |
| Approach: | They propose a method that automatically derives VQA examples at volume by leveraging existing image-caption annotations combined with neural models for textual question generation. |
| Outcome: | The proposed method improves state-of-the-art zero-shot accuracy by double digits and achieves robustness that lacks in the same model trained on human-annotated VQA data. |
CrossVQA: Scalably Generating Benchmarks for Systematically Testing VQA Generalization (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing visual question answering models leverage spurious biases and take shortcuts to improve performance. |
| Approach: | They propose a semi-automatic framework for generating disentangled shifts by introducing a controllable visual question-answer generation module that generates highly-relevant question-announcer pairs with the desired dataset style. |
| Outcome: | The proposed framework generates highly-relevant and diverse question-answer pairs with the desired dataset style. |
Open-Ended Visual Question Answering by Multi-Modal Domain Adaptation (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to visual question answering (VQA) are not suitable for real-world applications. |
| Approach: | They propose a supervised multi-modal domain adaptation method for visual question answering in images that exploits supervised domain adaptation. |
| Outcome: | The proposed method outperforms state-of-the-art methods on the benchmark VQA 2.0 and VizWiz datasets. |
ProtoVQA: An Adaptable Prototypical Framework for Explainable Fine-Grained Visual Question Answering (2025.emnlp-main)
Copied to clipboard
| Challenge: | Visual Question Answering (VQA) is increasingly used in diverse applications where models must provide accurate answers and explanations that humans can easily understand and verify. |
| Approach: | They propose a unified prototypical framework that learns question-aware prototypes that serve as reasoning anchors and applies spatially constrained matching to ensure that the selected evidence is coherent and semantically relevant. |
| Outcome: | The proposed framework yields faithful, fine-grained explanations while maintaining competitive accuracy. |
Uncovering the Full Potential of Visual Grounding Methods in VQA (2024.acl-long)
Copied to clipboard
| Challenge: | Visual Grounding (VG) methods in VQA aim to strengthen a model's reliance on question-relevant visual information. |
| Approach: | They propose to strengthen a model's reliance on question-relevant visual information by using a visual grounding method that is based on a question-related visual input. |
| Outcome: | The proposed methods can be much more effective when evaluation conditions are corrected. |
Sunny and Dark Outside?! Improving Answer Consistency in VQA through Entailed Question Generation (D19-1)
Copied to clipboard
| Challenge: | interacting with a model for Visual Question Answering (VQA) quickly reveals that these models lack consistency. |
| Approach: | They propose a dataset, ConVQA, and metrics that enable quantitative evaluation of consistency in VQA. |
| Outcome: | The proposed data augmentation module improves the consistency of VQA models on the Con-VQA dataset and is a strong baseline for future research. |