Papers by Arnab Mondal
Image Manipulation via Multi-Hop Instructions - A New Dataset and Weakly-Supervised Neuro-Symbolic Approach (2023.emnlp-main)
Copied to clipboard
Harman Singh, Poorva Garg, Mohit Gupta, Kevin Shah, Ashish Goswami, Satyam Modi, Arnab Mondal, Dinesh Khandelwal, Dinesh Garg, Parag Singla
| Challenge: | Recent studies have shown that neuro-symbolic models lack interpretability and are not robust to noise. |
| Approach: | They propose to extend Neuro Symbolic Concept Learning (NSCL) which has been quite effective for the task of Visual Question Answering (VQA) they create a new dataset for the image manipulation task and demonstrate that NeuroSIM is highly competitive with or beats SOTA baselines that make use of supervised data for manipulation. |
| Outcome: | The proposed system performs complex multi-hop reasoning over multi-object scenes and only requires weak supervision in the form of annotated data for VQA. |