Modal-specific Pseudo Query Generation for Video Corpus Moment Retrieval (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have shown promising results in video corpus moment retrieval . however, they relied on the expensive query annotations for the VCMR . |
| Approach: | They propose a self-supervised learning framework to localize video corpus moment without annotations. |
| Outcome: | The proposed framework can localize the video corpus moment without any explicit annotation on TVR dataset. |
Similar Papers
Cross-Lingual Cross-Modal Consolidation for Effective Multilingual Video Corpus Moment Retrieval (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing multilingual video corpus moment retrieval methods are based on a two-stream structure. |
| Approach: | They propose a multilingual video corpus moment retrieval task that uses a two-stream structure to generate a query-visual similarity and a subtitle stream exploits the query-subtitle similarity. |
| Outcome: | The proposed method improves accuracy on a large-scale video corpus moment retrieval dataset. |
mTVR: Multilingual Moment Retrieval in Videos (2021.acl-short)
Copied to clipboard
| Challenge: | mTVR is a multilingual video moment retrieval dataset with 218K queries in English and Chinese . Various datasets have been proposed or adapted for the task, but they are all created for a single language (English). |
| Approach: | They propose a multilingual video moment retrieval dataset with 218K queries from 21.8K TV show video clips. |
| Outcome: | The proposed model outperforms strong monolingual baselines while using fewer parameters. |
MS-DETR: Natural Language Video Localization with Sampling Moment-Moment Interaction (2023.acl-long)
Copied to clipboard
| Challenge: | Natural language video localization (NLVL) aims to localize a temporal moment from an untrimmed video that semantically corresponds to a given text query. |
| Approach: | They propose a proposal-based solution that generates proposals and selects the best matching proposal. |
| Outcome: | The proposed solution is faster than existing approaches on three public datasets. |
See More, Store Less: Memory-Efficient Resolution for Video Moment Retrieval (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing video moment retrieval methods rely on sparse frame sampling, risking information loss. |
| Approach: | a new video-based framework enhances memory efficiency while maintaining high information resolution . SMORE uses query-guided captions to encode semantics aligned with user intent . |
| Outcome: | a new framework improves memory efficiency while maintaining high information resolution . it achieves state-of-the-art performance on QVHighlights, Charades-STA, and ActivityNet-Captions benchmarks . |
ViLL-E: Video LLM Embeddings for Retrieval (2026.acl-long)
Copied to clipboard
| Challenge: | Video Large Language Models excel at video understanding tasks where outputs are textual . however, they underperform specialized embedding-based models in Retrieval tasks . |
| Approach: | They propose a video-LLM-based model with an embedding generation mechanism that allows the model to "think longer" for complex videos and stop early for easy ones. |
| Outcome: | The proposed model outperforms specialized embedding-based models in video understanding tasks while remaining competitive on VideoQA tasks. |
Reasoning Step-by-Step: Temporal Sentence Localization in Videos via Deep Rectification-Modulation Network (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods for temporal sentence localization in videos focus on visual content, but they are insufficient to model complex video contents. |
| Approach: | They propose a deep rectification-modulation network to correct attention misalignment . they use sentence information to capture frame-to-frame relation . |
| Outcome: | The proposed method achieves state-of-the-art performance on three public datasets. |
VChain: Chain-of-Visual-Thought for Reasoning in Video Generation (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent video generation models struggle to synthesize complex dynamics with a coherent chain of consequences. |
| Approach: | They propose a framework that injects visual reasoning signals from multimodal models into video generation. |
| Outcome: | a new framework that leverages multimodal models to generate sparse keyframes significantly improves quality of generated videos. |
VideoRAG: Retrieval-Augmented Generation over Video Corpus (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to generating models rely on text and images, but video content is a rich source of multimodal knowledge. |
| Approach: | They propose a framework that dynamically retrieves videos based on their relevance with queries . they use large video language models to represent video content for retrieval . |
| Outcome: | The proposed framework retrieves videos based on relevance with queries and integrates both visual and textual information. |
Localizing Moments in Video with Temporal Language (D18-1)
Copied to clipboard
| Challenge: | a novel model for localizing moments in a longer video using natural language queries is challenging . moment localization is similar to other language and vision tasks, but it offers an interesting opportunity to model temporal dependencies and reasoning in text. |
| Approach: | They propose a model that explicitly reasons about different temporal segments in a video . their dataset includes a dataset with real videos and template sentences . |
| Outcome: | The proposed model explicitly reasons about different temporal segments in a video . it shows that temporal context is important for localizing phrases which include temporal language . |
Generating Structured Pseudo Labels for Noise-resistant Zero-shot Video Sentence Localization (2023.acl-long)
Copied to clipboard
| Challenge: | Existing zero-shot pipelines generate event proposals and then generate a pseudo query for each event proposal. |
| Approach: | They propose a Structure-based Pseudo Label generation (SPL) that generates free-form interpretable pseudo queries before constructing query-dependent event proposals. |
| Outcome: | The proposed method learns with only video data without any annotation . it generates free-form interpretable pseudo queries before constructing query-dependent event proposals . |