Papers with DiDeMo
Fine-grained Semantic Alignment Network for Weakly Supervised Temporal Language Grounding (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing weakly supervised methods for temporal language grounding lose the complexity of the video and the semantics of the sentence. |
| Approach: | They propose a candidate-free framework for weakly supervised Temporal Language Grounding . they use a token-by-clip cross-modal semantic alignment module to learn alignment . |
| Outcome: | The proposed framework achieves state-of-the-art on two widely-used benchmarks: ActivityNet-Captions, and DiDeMo. |
RaP: Redundancy-aware Video-language Pre-training for Text-Video Retrieval (2022.findings-emnlp)
Copied to clipboard
| Challenge: | sparse sampling of videos suffers from inter-modal redundancy and visual redundancies . et al., 2021) proposes to sparsestly sample frames from videos to alleviate temporal redundance . |
| Approach: | They propose to use sparse sampling to alleviate temporal redundancy in videos . they propose to penalize high-redundant video patches and text tokens . |
| Outcome: | The proposed method improves on four benchmark datasets. |
Mitigating the Discrepancy Between Video and Text Temporal Sequences: A Time-Perception Enhanced Video Grounding method for LLM (2025.coling-main)
Copied to clipboard
| Challenge: | Existing video LLMs excel at capturing the overall description of a video but lack the ability to demonstrate an understanding of temporal dynamics and localized content within the video. |
| Approach: | They propose a Time-Perception Enhanced Video Grounding via Boundary Perception and Temporal Reasoning to improve LLMs' understanding of video temporality. |
| Outcome: | The proposed method improves on three datasets: ActivityNet, Charades, and DiDeMo (up to 11.2% improvement on R@0.3). |
Unifying Latent and Lexicon Representations for Effective Video-Text Retrieval (2024.lrec-main)
Copied to clipboard
Haowei Liu, Yaya Shi, Haiyang Xu, Chunfeng Yuan, Qinghao Ye, Chenliang Li, Ming Yan, Ji Zhang, Fei Huang, Bing Li, Weiming Hu
| Challenge: | Existing methods for video-text retrieval capture fine-grained semantic concepts . however, they lack the ability to capture finer-grain concepts such as objects and actions. |
| Approach: | They propose a dual-encoder architecture for fast video-text retrieval that learns lexicon representations to capture fine-grained semantics. |
| Outcome: | The proposed framework outperforms existing methods with 4.8% and 8.2% improvement on MSR-VTT and DiDeMo respectively. |