Papers by Ayush Shrivastava
VISITRON: Visual Semantics-Aligned Interactively Trained Object-Navigator (2022.findings-acl)
Copied to clipboard
Ayush Shrivastava, Karthik Gopalakrishnan, Yang Liu, Robinson Piramuthu, Gokhan Tur, Devi Parikh, Dilek Hakkani-Tur
| Challenge: | Interactive robots navigating photo-realistic environments need to be trained to handle dynamic nature of dialogue and vision-and-language navigation (VLN). |
| Approach: | They propose a Transformer-based multi-modal navigator that is better suited to the interactive regime inherent to Cooperative Vision-and-Dialog Navigation (CVDN). |
| Outcome: | The proposed model is trained to identify and associate object-level concepts and semantics between the environment and dialogue history and identify when to interact vs. navigate via imitation learning of a binary classification head. |