Where is this coming from? Making groundedness count in the evaluation of Document VQA models (2025.findings-naacl)
Copied to clipboard
| Challenge: | Document Visual Question Answering (VQA) models have come close to or matching human performance on some benchmarks. |
| Approach: | They propose a method that accounts for the semantic and multimodal groundedness of a model’s outputs and can be parameterized so that users can configure the score according to their preferences. |
| Outcome: | The proposed method produces scores that are a better indicator of a model’s robustness and tends to give higher rewards to better-calibrated answers. |
Similar Papers
Uncovering the Full Potential of Visual Grounding Methods in VQA (2024.acl-long)
Copied to clipboard
| Challenge: | Visual Grounding (VG) methods in VQA aim to strengthen a model's reliance on question-relevant visual information. |
| Approach: | They propose to strengthen a model's reliance on question-relevant visual information by using a visual grounding method that is based on a question-related visual input. |
| Outcome: | The proposed methods can be much more effective when evaluation conditions are corrected. |
MAGIC-VQA: Multimodal And Grounded Inference with Commonsense Knowledge for Visual Question Answering (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing Large Vision-Language Models (LVLMs) lack integrated commonsense knowledge . lack of integrated common knowledge limits their robustness and accuracy in VQA . |
| Approach: | They propose a framework to enhance multimodal inference by integrating commonsense reasoning. |
| Outcome: | MAGIC-VQA improves comprehensive benchmark datasets, surpassing existing models in tasks requiring advanced commonsense reasoning. |
‘Just because you are right, doesn’t mean I am wrong’: Overcoming a bottleneck in development and evaluation of Open-Ended VQA tasks (2021.eacl-main)
Copied to clipboard
Man Luo, Shailaja Keyur Sampat, Riley Tallman, Yankai Zeng, Manuha Vancha, Akarshan Sajja, Chitta Baral
| Challenge: | Existing visual question answering datasets assume only one ground truth answer for each question. |
| Approach: | They propose alternative answer sets (AAS) of ground-truth answers to address this limitation . they modify top VQA solvers to support multiple plausible answers for a question . |
| Outcome: | The proposed approach improves on the GQA dataset and shows that it is more efficient than previous approaches. |
Measuring Faithful and Plausible Visual Grounding in VQA (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Lack of visual grounding (VG) in VQA systems can manifest in over-reliance on irrelevant image parts or a disregard for the visual modality entirely. |
| Approach: | They propose a new metric that measures if a model identifies question-relevant objects in the scene and relies on the information contained in the relevant objects when producing its answer. |
| Outcome: | The proposed metric measures if a model identifies question-relevant objects in the scene and relies on the information contained in the relevant objects when producing its answer. |
Grounded, or a Good Guesser? A Per-Question Balanced Dataset to Separate Blind from Grounded Models for Embodied Question Answering (2025.acl-short)
Copied to clipboard
Miles Shelton, Nate Wingerd, Kritim K Rijal, Ayush Garg, Adelina Gutic, Brett Barnes, Catherine Finegan-Dollak
| Challenge: | Embodied question answering (EQA) is based on using perception and action in an environment to answer natural language questions. |
| Approach: | They propose a "per-question balanced" EQA dataset that uses two different environments to ground a model's answers in its environment. |
| Outcome: | The proposed model performs better than chance on the PQB-EQA benchmark, showing that it does not require the model to use perception, let alone to act in its environment to find the answer. |
Fine-Grained Detection of Context-Grounded Hallucinations Using LLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing representations of hallucinations limit the types of errors that can be expressed, so we propose a new representation based on free-form textual descriptions, capturing the full range of possible errors. |
| Approach: | They propose a benchmark for localizing hallucinations using LLMs with a human annotation of over 1,000 examples and a protocol to verify its quality in a humans evaluation. |
| Outcome: | The proposed representation captures the full range of possible errors, and the best model achieves an F1 score of 0.67. |
Groundedness in Retrieval-augmented Long-form Generation: An Empirical Study (2024.findings-naacl)
Copied to clipboard
| Challenge: | a significant portion of correct answers remain compromised by hallucinations in large language models. |
| Approach: | They examine whether every generated sentence is grounded in retrieved documents or the model’s pre-training data. |
| Outcome: | The findings highlight the need for more robust mechanisms in large language models to mitigate the generation of ungrounded content. |
A negative case analysis of visual grounding methods for VQA (2020.acl-main)
Copied to clipboard
| Challenge: | Existing Visual Question Answering (VQA) methods exploit dataset biases and spurious statistical correlations instead of producing correct answers for the right reasons. |
| Approach: | They propose to incorporate visual cues to better ground VQA models . they also propose a regularization effect which prevents over-fitting to linguistic priors . |
| Outcome: | The proposed method outperforms existing methods on the Visual Question Answering (VQA) dataset. |
Can Visual Dialogue Models Do Scorekeeping? Exploring How Dialogue Representations Incrementally Encode Shared Knowledge (2022.acl-short)
Copied to clipboard
| Challenge: | Existing evaluation methods for visual dialogue models are not consistent . |
| Approach: | They propose a theory-based evaluation method to examine to what degree visual dialogue models incrementally build up representations that do scorekeeping. |
| Outcome: | The proposed method aims to determine to what degree models build up representations that are appropriate to do scorekeeping of shared commitments throughout a visual dialogue. |
Grounded Concreteness: Human-Like Concreteness Sensitivity in Vision–Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | a long tradition in cognitive science treats concreteness as a graded dimension of conceptual representation . concrete words benefit from richer sensory codes and exhibit robust behavioral advantages over abstract words . |
| Approach: | They compare vision-language models with text-only large language models to test their concreteness . they find that VLMs show more human-like sensitivity to concreteness than LLMs . |
| Outcome: | The proposed model-based training improves on the Llama text backbones and Llma Vision counterparts. |