Papers by Phillip Isola

Copied to clipboard

Challenge:	Existing approaches to vision-and-language navigation use visual features as the perceptual representation of a visual representation of an agent's egocentric panoramic view.
Approach:	They propose to use off-the-shelf vision systems to convert an agent’s egocentric panoramic view into natural language descriptions.
Outcome:	The proposed approach improves on the R2R VLN benchmark by using synthetic trajectories from a prompted language model and domain transfer where a policy learned on one simulated environment (ALFRED) is transferred to another (more realistic) environment and combining both vision- and language-based representations.