Towards Robust Visual Question Answering: Making the Most of Biased Samples via Contrastive Learning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies have shown that biased samples can be brittle for VQA models . however, the improvements on OOD data severely sacrifice the performance on the in-distribution (ID) data. |
| Approach: | They propose a contrastive learning approach that exploits biased samples for unbiased information that contributes to reasoning. |
| Outcome: | The proposed method achieves competitive performance on the OOD dataset while maintaining robustness on the ID dataset. |
Similar Papers
Learning to Contrast the Counterfactual Samples for Robust Visual Question Answering (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods of generating counterfactual samples are not fully utilized in the task of Visual Question Answering (VQA). |
| Approach: | They propose a self-supervised contrastive learning mechanism to learn the relationship between original samples, factual samples and counterfactual samples. |
| Outcome: | The proposed method surpasses state-of-the-art models on the VQA-CP dataset, a diagnostic benchmark for assessing the VQ model’s robustness. |
Digging out Discrimination Information from Generated Samples for Robust Visual Question Answering (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to solve this problem rely on additional annotations and generate negative samples . |
| Approach: | They propose a method to Dig out Discrimination information from Generated samples to address these limitations. |
| Outcome: | The proposed method improves on the visual question-answering datasets. |
A Multi-modal Debiasing Model with Dynamical Constraint for Robust Visual Question Answering (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have shown that many well-developed Visual Question Answering systems suffer from bias problem. |
| Approach: | They propose a way to mitigate bias problem by subtracting bias score from standard VQA base score. |
| Outcome: | The proposed method improves on the VQA v2.0 and VQA-CP V2,0 datasets. |
Dataset Bias Mitigation in Multiple-Choice Visual Question Answering and Beyond (2023.findings-emnlp)
Copied to clipboard
Zhecan Wang, Long Chen, Haoxuan You, Keyang Xu, Yicheng He, Wenhao Li, Noel Codella, Kai-Wei Chang, Shih-Fu Chang
| Challenge: | Existing studies have examined dataset biases in VQA benchmarks with short-phrase answers Multiple-choice Question with the LONG Answers (VCR, VLEP, etc.) |
| Approach: | They propose to use Adversarial Data Synthesis (ADS) to generate synthetic training and debiased evaluation data and introduce Intra-sample Counterfactual Training (ICT) to assist models in utilizing synthesized training data. |
| Outcome: | The proposed approach improves model performance even in domain-shifted scenarios. |
Overcoming Language Priors in Visual Question Answering via Distinguishing Superficially Similar Instances (2022.coling-1)
Copied to clipboard
| Challenge: | Existing VQA models rely on the superficial correlation between question type and frequent answers to make predictions, without really understanding the input. |
| Approach: | They propose a training framework that explicitly encourages the VQA model to distinguish between superficially similar instances. |
| Outcome: | The proposed framework achieves state-of-the-art performance on VQA-CP v2 . it explicitly encourages the model to distinguish between the superficially similar instances . |
Towards Robust Extractive Question Answering Models: Rethinking the Training Methodology (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models lack robustness against distribution shifts and adversarial attacks when training on unanswerable questions in EQA datasets. |
| Approach: | They propose a novel loss function for the EQA problem to improve the robustness of extractive question answering models by adding adversarial questions to a crowdsourcing process. |
| Outcome: | The proposed method maintains in-domain performance while improving on out-of-domain datasets. |
Being Negative but Constructively: Lessons Learnt from Creating Better Visual Question Answering Datasets (N18-1)
Copied to clipboard
| Challenge: | Visual question answering datasets are a form of (visual) Turing test that artificial intelligence should strive to achieve. |
| Approach: | They propose automatic procedures to remedy design deficiencies in visual question answering datasets . they propose to use a set of decoys to re-construct decoying answers for two popular Visual QA datasets. |
| Outcome: | The proposed procedures improve the performance of the proposed datasets. |
A negative case analysis of visual grounding methods for VQA (2020.acl-main)
Copied to clipboard
| Challenge: | Existing Visual Question Answering (VQA) methods exploit dataset biases and spurious statistical correlations instead of producing correct answers for the right reasons. |
| Approach: | They propose to incorporate visual cues to better ground VQA models . they also propose a regularization effect which prevents over-fitting to linguistic priors . |
| Outcome: | The proposed method outperforms existing methods on the Visual Question Answering (VQA) dataset. |
Compressing and Debiasing Vision-Language Pre-Trained Models for Visual Question Answering (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on VQA models have found that they suffer from dataset biases and inefficient memory footprints. |
| Approach: | They investigate whether a VLP can be compressed and debiased simultaneously by searching sparse and robust subnetworks. |
| Outcome: | The proposed compression and debiasing pipelines outperform the debiased full VLPs on VQA tasks. |
FVQA 2.0: Introducing Adversarial Samples into Fact-based Visual Question Answering (2023.findings-eacl)
Copied to clipboard
| Challenge: | Fact-based Visual Question Answering (FVQA) is a visual question answering task that requires information retrieval using common sense knowledge graphs to answer. |
| Approach: | They propose a new test question with adversarial variants to address this imbalance by using a KB-VQA dataset that is small and contains only one answer per question. |
| Outcome: | The proposed version reduces the vulnerability of the original FVQA dataset without human annotations. |