A Gaze-grounded Visual Question Answering Dataset for Clarifying Ambiguous Japanese Questions (2024.lrec-main)
Copied to clipboard
| Challenge: | Visual question answering (VQA) tasks are often based on directives, which can cause ambiguities in human utterances. |
| Approach: | They propose a method that clarifies ambiguous questions using gaze information . they propose combining gaze information with gaze information to improve accuracy . |
| Outcome: | The proposed method improves performance in some cases of a GazeVQA system on Gaze. |
Similar Papers
Teaching Vision-Language Models to Ask: Resolving Ambiguity in Visual Questions (2025.acl-long)
Copied to clipboard
| Challenge: | Existing research addresses ambiguous visual questions by rephrasing questions, but it fails to address the inherently interactive nature of user interactions with visual language models (VLMs). Existing studies focus on re-phrase questions, and lack of a benchmark to assess VLMs’ capacity for resolving ambiguities through interaction. |
| Approach: | They propose a visual question answering task that provides a natural language answer to a question based on a given image and an automated pipeline to generate ambiguity-clarification question pairs. |
| Outcome: | The proposed benchmark targets three common categories of ambiguity in visual question answering (VQA) context and encompasses various VQA scenarios. |
A Corpus for Visual Question Answering Annotated with Frame Semantic Information (2020.lrec-1)
Copied to clipboard
| Challenge: | Visual Question Answering (VQA) is a computer vision problem. |
| Approach: | They propose to annotate a visual question answering dataset with verb semantics to help the model understand verbs. |
| Outcome: | The proposed system is built on the imSitu dataset annotated with verb semantic information. |
CommVQA: Situating Visual Question Answering in Communicative Contexts (2024.emnlp-main)
Copied to clipboard
| Challenge: | Current visual question answering models are trained on image-question pairs in isolation, but the questions people ask are dependent on their informational needs and prior knowledge about the image content. |
| Approach: | They propose a visual question-answer-as-question dataset that contains 1000 images and 8,949 question-announcer pairs to evaluate how situating images within naturalistic contexts shapes visual questions. |
| Outcome: | The proposed dataset contains 1000 images and 8,949 question-answer pairs. |
GazeVQA: A Video Question Answering Dataset for Multiview Eye-Gaze Task-Oriented Collaborations (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on the use of exocentric and egocentric videos in video question answering are focusing on eye-gaze information. |
| Approach: | They propose a task-oriented VQA dataset that captures eye-gaze information . they propose assisting models that ground the perceptual input into semantic information based on three different answer types . |
| Outcome: | The proposed model can ground the perceptual input into semantic information while reducing ambiguities. |
Visual Question Answering Dataset for Bilingual Image Understanding: A Study of Cross-Lingual Transfer Using Attention Maps (C18-1)
Copied to clipboard
| Challenge: | Existing literature on visual question answering (VQA) focuses on English, but there are no datasets for other languages. |
| Approach: | They propose a cross-lingual method to make use of English annotation to improve Japanese VQA . they use attention maps generated from English questions to improve the task . |
| Outcome: | The proposed method performs better than using a monolingual corpus in Japanese than using monolingual ones. |
CondAmbigQA: A Benchmark and Dataset for Conditional Ambiguous Question Answering (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) generate unreliable responses due to their cognitive alignment of context and intent. |
| Approach: | They propose a benchmark to identify possible implicit assumptions in QA questions . they use retrieved Wikipedia fragments to identify interpretations for a given query . |
| Outcome: | The proposed benchmark identifies possible implicit assumptions and improves answer accuracy by 11.75% . retrieved Wikipedia fragments help identify possible interpretations for a given query . |
A negative case analysis of visual grounding methods for VQA (2020.acl-main)
Copied to clipboard
| Challenge: | Existing Visual Question Answering (VQA) methods exploit dataset biases and spurious statistical correlations instead of producing correct answers for the right reasons. |
| Approach: | They propose to incorporate visual cues to better ground VQA models . they also propose a regularization effect which prevents over-fitting to linguistic priors . |
| Outcome: | The proposed method outperforms existing methods on the Visual Question Answering (VQA) dataset. |
Why Did the Chicken Cross the Road? Rephrasing and Analyzing Ambiguous Questions in VQA (2023.acl-long)
Copied to clipboard
| Challenge: | Visual question answering models seek to answer questions about images . ambiguity can exist at all levels of linguistic analysis, but disagreements can be difficult to detect and resolve . |
| Approach: | They develop a question-generation model which integrates group information without supervision and uses a dataset of ambiguous examples to annotate answers. |
| Outcome: | The proposed model can integrate answer group information without supervision and is able to fill knowledge gaps and convey requests. |
Looking Beyond the One: Operationalizing and Eliciting Visual Ambiguity in VLLMs (2026.acl-long)
Copied to clipboard
| Challenge: | Visual question answering systems typically collapse ambiguity, committing to a single interpretation during decoding and evaluation. |
| Approach: | They operationalize ambiguity as the existence of multiple answer-supporting regions in an image . they show that ambiguities are already encoded in their internal representations . |
| Outcome: | The proposed approach makes ambiguity observable without exhaustive annotations . ambiguities are already encoded in models, but not reliably expressed in outputs despite hidden states . |
Overcoming Language Priors in Visual Question Answering via Distinguishing Superficially Similar Instances (2022.coling-1)
Copied to clipboard
| Challenge: | Existing VQA models rely on the superficial correlation between question type and frequent answers to make predictions, without really understanding the input. |
| Approach: | They propose a training framework that explicitly encourages the VQA model to distinguish between superficially similar instances. |
| Outcome: | The proposed framework achieves state-of-the-art performance on VQA-CP v2 . it explicitly encourages the model to distinguish between the superficially similar instances . |