Annotations Are Not All You Need: A Cross-modal Knowledge Transfer Network for Unsupervised Temporal Sentence Grounding (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work on temporal sentence grounding rely on expensive video-query paired annotations . despite this, there are no ground-truth annotations in the current work . |
| Approach: | They propose to use paired video-query and segment boundary annotations to generate temporal sentence grounding without training. |
| Outcome: | The proposed model outperforms existing unsupervised methods and beats supervised ones on two challenging datasets. |
Similar Papers
Weakly-Supervised Temporal Article Grounding (2022.emnlp-main)
Copied to clipboard
Long Chen, Yulei Niu, Brian Chen, Xudong Lin, Guangxing Han, Christopher Thomas, Hammad Ayyubi, Heng Ji, Shih-Fu Chang
| Challenge: | Existing VG models make unrealistic assumptions about how to ground video segments . a recent study has shown that video grounding can be useful for downstream applications . |
| Approach: | They propose a new task: Weakly-Supervised temporal Article Grounding (WSAG) given an article and a relevant video, WSAG aims to localize all "groundable" sentences to the video. |
| Outcome: | The proposed method is simple but effective, and it can be used in real-world applications. |
Rethinking the Video Sampling and Reasoning Strategies for Temporal Sentence Grounding (2022.findings-emnlp)
Copied to clipboard
Jiahao Zhu, Daizong Liu, Pan Zhou, Xing Di, Yu Cheng, Song Yang, Wenzheng Xu, Zichuan Xu, Yao Wan, Lichao Sun, Zeyu Xiong
| Challenge: | Existing methods for temporal sentence grounding ignore two crucial issues . 1) Boundary-bias: the video downsampling process may lose these two frames . 2) Reasoning-biases: such incorrect new boundary frames lead to the reasoning bias . |
| Approach: | They propose a siamese sampling mechanism to generate additional contextual frames . they use a reasoning strategy to learn the inter-relationship among these frames a . |
| Outcome: | Extensive experiments demonstrate the effectiveness of a new siamese sampling network on three challenging datasets. |
Towards Parameter-Efficient Integration of Pre-Trained Language Models In Temporal Video Grounding (2023.findings-acl)
Copied to clipboard
Erica Kido Shimomoto, Edison Marrese-Taylor, Hiroya Takamura, Ichiro Kobayashi, Hideki Nakayama, Yusuke Miyao
| Challenge: | Recent studies have improved query inputs with pre-trained language models, but the effects of this integration are unclear. |
| Approach: | They propose to integrate query sentences with pre-trained language models to train TVG models. |
| Outcome: | The proposed model integrates query sentences with pre-trained language models at cost of more expensive training. |
Progressively Guide to Attend: An Iterative Alignment Framework for Temporal Sentence Grounding (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to learn effective alignment between vision and language features are insufficient in practice due to complicated multi-step reasoning. |
| Approach: | They propose an iterative alignment network which iterates inter- and intra-modal features within multiple steps for more accurate grounding. |
| Outcome: | The proposed model performs better than the state-of-the-arts on three challenging benchmarks. |
Weakly-Supervised Spatio-Temporally Grounding Natural Sentence in Video (P19-1)
Copied to clipboard
| Challenge: | Existing techniques for weakly-supervised spatio-temporally grounding natural sentence in video are lacking . |
| Approach: | They propose a weakly-supervised task for spatially grounding sentences in video . they extract instances from video and encode them using attentive interactor . results demonstrate superiority of their proposed task over baseline approaches . |
| Outcome: | The proposed model outperforms baseline approaches in a weakly-supervised task . it can characterize reliable instance-sentence pairs and penalize unreliable ones . |
Read Before Grounding: Scene Knowledge Visual Grounding via Multi-step Parsing (2025.coling-main)
Copied to clipboard
| Challenge: | Existing VG datasets use simple textual descriptions with limited attribute and spatial information between images and text. |
| Approach: | They propose a method that transforms visual knowledge into concise, information-dense visual descriptions. |
| Outcome: | The proposed method significantly improves performance of multimodal grounding models. |
Fine-grained Semantic Alignment Network for Weakly Supervised Temporal Language Grounding (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing weakly supervised methods for temporal language grounding lose the complexity of the video and the semantics of the sentence. |
| Approach: | They propose a candidate-free framework for weakly supervised Temporal Language Grounding . they use a token-by-clip cross-modal semantic alignment module to learn alignment . |
| Outcome: | The proposed framework achieves state-of-the-art on two widely-used benchmarks: ActivityNet-Captions, and DiDeMo. |
Learning to Focus on the Foreground for Temporal Sentence Grounding (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods for temporal sentence grounding do not capture subtle details of small objects. |
| Approach: | They propose a detection-free framework for temporal sentence grounding that learns to locate foreground regions related to the query in consecutive frames. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on three challenging datasets. |
Reasoning Step-by-Step: Temporal Sentence Localization in Videos via Deep Rectification-Modulation Network (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods for temporal sentence localization in videos focus on visual content, but they are insufficient to model complex video contents. |
| Approach: | They propose a deep rectification-modulation network to correct attention misalignment . they use sentence information to capture frame-to-frame relation . |
| Outcome: | The proposed method achieves state-of-the-art performance on three public datasets. |
Mitigating the Discrepancy Between Video and Text Temporal Sequences: A Time-Perception Enhanced Video Grounding method for LLM (2025.coling-main)
Copied to clipboard
| Challenge: | Existing video LLMs excel at capturing the overall description of a video but lack the ability to demonstrate an understanding of temporal dynamics and localized content within the video. |
| Approach: | They propose a Time-Perception Enhanced Video Grounding via Boundary Perception and Temporal Reasoning to improve LLMs' understanding of video temporality. |
| Outcome: | The proposed method improves on three datasets: ActivityNet, Charades, and DiDeMo (up to 11.2% improvement on R@0.3). |