Papers by Abhishek Das
Connecting Language and Vision to Actions (P18-5)
Copied to clipboard
| Challenge: | Recent advances in language and vision have made incredible progress in describing images and interacting with visual content in a physical or embodied environment. |
| Approach: | This tutorial will provide an overview of the growing number of multimodal tasks and datasets that combine textual and visual understanding. |
| Outcome: | This tutorial will review the state-of-the-art approaches to selected tasks such as image captioning, visual question answering and visual dialog. |
Multi-step Entity-centric Information Retrieval for Multi-Hop Question Answering (D19-58)
Copied to clipboard
Rajarshi Das, Ameya Godbole, Dilip Kavarthapu, Zhiyu Gong, Abhishek Singhal, Mo Yu, Xiaoxiao Guo, Tian Gao, Hamed Zamani, Manzil Zaheer, Andrew McCallum
| Challenge: | Multi-hop question answering (QA) requires an information retrieval system that can find multiple supporting evidence needed to answer the question. |
| Approach: | They propose a technique that uses information of entities present in the initial retrieved evidence to learn to ‘hop’ onto other relevant evidence. |
| Outcome: | The proposed method boosts retrieval performance on a multi-hop question answering dataset with 5 million Wikipedia paragraphs and a model without training increases its performance by 10.59 F1. |
Improving Generative Visual Dialog by Answering Diverse Questions (D19-1)
Copied to clipboard
| Challenge: | Prior work on training generative Visual Dialog models with reinforcement learning has shown that this improvement saturates and starts degrading after a few rounds of interaction, and does not lead to a better Visual Dialog model. |
| Approach: | They propose a Q-Bot-A-Bot image-guessing game that allows Q-BOT to ask diverse questions, thus reducing repetitions and enabling A-BOTT to explore a larger state space during RL. |
| Outcome: | The proposed approach improves Q-Bot-A-Bot image-guessing performance but degrades after a few rounds of interaction and does not lead to a better Visual Dialog model. |