Intrinsic Subgraph Generation for Interpretable Graph Based Visual Question Answering (2024.lrec-main)
Copied to clipboard
| Challenge: | Visual Question Answering (VQA) is acknowledged as a challenging multi-modal task for Machine Learning (ML). |
| Approach: | They propose an interpretable approach for graph-based Visual Question Answering . their model is designed to intrinsically produce a subgraph during the question-answering process as its explanation . |
| Outcome: | The proposed model outperforms existing explainable methods on a graph-based VQA dataset. |
Similar Papers
Discrete Subgraph Sampling for Interpretable Graph based Visual Question Answering (2025.coling-main)
Copied to clipboard
| Challenge: | XAI aims to make machine learning models more transparent, but interpretable approaches are relatively rare. |
| Approach: | They integrate discrete subset sampling methods into a graph-based visual question answering system to evaluate their interpretability. |
| Outcome: | The proposed methods mitigate trade-off between interpretability and answer accuracy while achieving strong co-occurrences between answer and question tokens. |
Tell-and-Answer: Towards Explainable Visual Question Answering using Attributes and Captions (D18-1)
Copied to clipboard
| Challenge: | Existing approaches to visual question answering represent images using pre-trained CNNs . but they rarely provide any insight, apart from the answer, into the VQA process . |
| Approach: | They propose to break up the end-to-end VQA into two steps: explaining and reasoning . they first extract attributes and generate descriptions as explanations for an image . a reasoning module utilizes these explanations in place of the image to infer an answer . |
| Outcome: | The proposed system achieves comparable performance with baselines, but with added benefits of explanability and the ability to improve with higher quality explanations. |
Towards Knowledge-Augmented Visual Question Answering (2020.coling-main)
Copied to clipboard
| Challenge: | Visual Question Answering (VQA) is a challenging task for humans, but it is effortless for visual-based approaches. |
| Approach: | They propose a visual-based approach that captures interactions between visual scenes and external knowledge sources and exploits ConceptNet as the source of general knowledge. |
| Outcome: | The proposed model learns a question-adaptive graph representation of related knowledge instances. |
RG-VQA: Leveraging Retriever-Generator Pipelines for Knowledge Intensive Visual Question Answering (2025.findings-emnlp)
Copied to clipboard
Settaluri Lakshmi Sravanthi, Pulkit Agarwal, Debjyoti Mondal, Rituraj Singh, Subhadarshi Panda, Ankit Mishra, Kiran Pradeep, Srihari K B, Godawari Sudhakar Rao, Pushpak Bhattacharyya
| Challenge: | Existing methods to improve the reasoning capabilities of VQA systems are limited due to complexity of graph neural networks and end-to-end training. |
| Approach: | They propose a method to integrate Dense Passage Retrievers with Vision Language Models to boost the reasoning capabilities of VQA systems. |
| Outcome: | The proposed method outperforms human accuracy and GPT-4 in the ScienceQA dataset. |
Do explanations make VQA models more predictable to a human? (D18-1)
Copied to clipboard
| Challenge: | Existing explanations of a model's behavior are not used in interactive tasks like Visual Question Answering (VQA). |
| Approach: | They analyze existing explanations and their role in making a VQA model more predictable to a human by using human-in-the-loop approaches that treat the model as a black-box. |
| Outcome: | The proposed explanations make a model more predictable to humans, whereas human-in-the-loop approaches treat it as a black-box do. |
Charting the Future: Using Chart Question-Answering for Scalable Evaluation of LLM-Driven Data Visualizations (2025.coling-main)
Copied to clipboard
| Challenge: | Existing evaluation methods rely on human judgment to assess data accuracy and visual communication, which is costly and unscalable. |
| Approach: | They propose a framework that leverages Visual Question Answering (VQA) models to automate the evaluation of LLM-generated data visualizations. |
| Outcome: | The proposed framework assesses data representation quality and communicative clarity of charts using two leading VQA benchmark datasets, ChartQA and PlotQA, with visualizations generated by OpenAI’s GPT-3.5 Turbo and Meta’s Llama 3.1 70B-Instruct models. |
All You May Need for VQA are Image Captions (2022.naacl-main)
Copied to clipboard
| Challenge: | Visual Question Answering (VQA) has benefited from increasingly sophisticated models, but has not enjoyed the same level of engagement in terms of data creation. |
| Approach: | They propose a method that automatically derives VQA examples at volume by leveraging existing image-caption annotations combined with neural models for textual question generation. |
| Outcome: | The proposed method improves state-of-the-art zero-shot accuracy by double digits and achieves robustness that lacks in the same model trained on human-annotated VQA data. |
ConceptBert: Concept-Aware Representation for Visual Question Answering (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Visual Question Answering (VQA) is a challenging task that has received increasing attention from both the computer vision and the natural language processing communities. |
| Approach: | They propose an algorithm which learns a joint Concept-Vision-Language embedding for questions which require common sense knowledge from external structured content. |
| Outcome: | The proposed model is based on the Outer Knowledge-VQA and VQA datasets. |
ReasoningLM: Enabling Structural Subgraph Reasoning in Pre-trained Language Models for Question Answering over Knowledge Graph (2023.emnlp-main)
Copied to clipboard
| Challenge: | Question Answering over Knowledge Graph (KGQA) aims to find answer entities for natural language questions based on knowledge graphs. |
| Approach: | They propose a subgraph-aware self-attention mechanism to imitate the graph neural network (GNN) based module to perform multi-hop reasoning on KG. |
| Outcome: | The proposed method surpasses state-of-the-art models by a large margin even with fewer updated parameters and less training data. |
Explanation Graph Generation via Generative Pre-training over Synthetic Graphs (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing frameworks for explanation graph generation are limited due to the large number of datasets available. |
| Approach: | They propose a text-to-graph generative task to pre-train a model to bridge the text-graph gap. |
| Outcome: | The proposed framework surpasses all baseline systems with remarkable margins on ExplaGraphs and CommonsenseQA. |