Papers with R4R
Stay on the Path: Instruction Fidelity in Vision-and-Language Navigation (P19-1)
Copied to clipboard
| Challenge: | Existing metrics for vision-and-language navigation focus on goal completion rather than the sequence of actions corresponding to the instructions. |
| Approach: | They propose to use a room-to-room dataset to measure the length of instruction followed by agents. |
| Outcome: | The proposed metric outperforms existing metrics for Room-to-Room tasks because it is direct-to goal shortest. |
DELAN: Dual-Level Alignment for Vision-and-Language Navigation by Cross-Modal Contrastive Learning (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing studies focus on cross-modal attention at the fusion stage, but modality features generated by disparate uni-encoders reside in their own spaces, leading to a decline in the quality of cross-modulation and decision-making. |
| Approach: | They propose a framework to align navigation-related modalities before fusion by cross-modal contrastive learning. |
| Outcome: | The proposed framework integrates with the majority of existing models, resulting in improved navigation performance on various VLN benchmarks, including R2R, R4R, and CVDN. |
LOViS: Learning Orientation and Visual Signals for Vision and Language Navigation (2022.coling-1)
Copied to clipboard
| Challenge: | Existing Transformer-based VLN agents entangle orientation and vision information, which limits the learning of each information source. |
| Approach: | They propose to design a navigation agent with explicit Orientation and Vision modules . they use a set of pre-training tasks to feed the modules into the model . |
| Outcome: | The proposed model improves on R2R and R4R datasets and achieves state-of-the-art results. |
VLN-Trans: Translator for the Vision and Language Navigation Agent (2023.acl-long)
Copied to clipboard
| Challenge: | We observe two kinds of instructions that make the grounding in the vision-and-language navigation task quite challenging. |
| Approach: | They propose to use a translator module to convert instructions into easy-to-follow sub-instruction representations at each step. |
| Outcome: | The proposed model is based on a Room2Room (R2R), Room4room (R4R), and Room2room Last (R1R-Last) datasets and achieves state-of-the-art results on multiple benchmarks. |