Papers by Sameer Dharur
SOrT-ing VQA Models : Contrastive Gradient Learning for Improved Consistency (2021.naacl-main)
Copied to clipboard
| Challenge: | Current visual question answering models are inconsistent in their understanding of the world . they answer difficult reasoning questions correctly but get associated sub-questions wrong . |
| Approach: | They propose a gradient-based interpretability approach to determine the questions most strongly correlated with the reasoning question on an image and a contrastive gradient learning based approach called Sub-question Oriented Tuning (SOrT). |
| Outcome: | The proposed approach improves model consistency by up to 6.5% points over existing approaches while improving visual grounding and robustness to rephrasings of questions. |