VideoINSTA: Zero-shot Long Video Understanding via Informative Spatial-Temporal Reasoning with LLMs (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Long video understanding presents unique challenges due to the complexity of reasoning over extended timespans. |
| Approach: | They propose a framework VideoINSTA to leverage large language models for video understanding . they propose 'event-based temporalreasoning' and 'content-based spatial reasoning' |
| Outcome: | The proposed model significantly improves state-of-the-art on three long video question-answering benchmarks. |
Similar Papers
TemporalVLM: Video LLMs for Temporal Reasoning in Long Videos (2026.findings-acl)
Copied to clipboard
| Challenge: | Several video understanding applications require the ability of temporal reasoning. |
| Approach: | They propose a video large language model for temporal reasoning and fine-grained understanding in long videos. |
| Outcome: | The proposed model outperforms existing methods in time and motion studies and temporal action segmentation evaluations. |
Detecting and Mitigating Challenges in Zero-Shot Video Summarization with Video LLMs (2025.findings-acl)
Copied to clipboard
Luca Cagliero, Lorenzo Vaiani, Eliana Pastor, Alkis Koudounas, Elena Baralis, Vittorio Mazzia, Sandro Pollastrini, Thomas Gueudre, Manuel Giollo, Daniele Amberti, Yue Wu
| Challenge: | Video Large Language Models (VLLMs) exhibit impressive zero-shot capabilities in video analysis, but their performance varies significantly depending on the LLM prompt, the characteristics of the video, and the properties of the training data and LLM architecture. |
| Approach: | They propose to use Chain-of-Thought prompting to inject knowledge extracted by external, lightweight models into video summarization benchmarks to evaluate their performance. |
| Outcome: | The proposed solutions improve summarization performance by injecting knowledge extracted by external, lightweight models. |
VideoPASTA: 7K Preference Pairs That Matter for Video-LLM Alignment (2025.emnlp-main)
Copied to clipboard
| Challenge: | Video-language models excel at understanding video content but struggle with spatial relationships, temporal ordering, and cross-frame continuity. |
| Approach: | They propose a framework that trains video-LLMs to distinguish accurate representations from carefully crafted adversarial examples. |
| Outcome: | Experiments show that VideoPASTA improves performance without human annotation or captioning . the framework can be used on various state-of-the-art video-LLMs with no human annotation . |
Enhancing Temporal Modeling of Video LLMs via Time Gating (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing Video Large Language Models neglect temporal information in video data, leading to struggles with temporal-aware video understanding. |
| Approach: | They propose a Time Gating Video LLM (TG-Vid) that employs a time gating module to enhance temporal modeling. |
| Outcome: | The proposed model outperforms existing Large Language Models on video-and-language tasks and ablation studies show that the model outpersforms the existing models. |
Mitigating the Discrepancy Between Video and Text Temporal Sequences: A Time-Perception Enhanced Video Grounding method for LLM (2025.coling-main)
Copied to clipboard
| Challenge: | Existing video LLMs excel at capturing the overall description of a video but lack the ability to demonstrate an understanding of temporal dynamics and localized content within the video. |
| Approach: | They propose a Time-Perception Enhanced Video Grounding via Boundary Perception and Temporal Reasoning to improve LLMs' understanding of video temporality. |
| Outcome: | The proposed method improves on three datasets: ActivityNet, Charades, and DiDeMo (up to 11.2% improvement on R@0.3). |
VideoQA-TA: Temporal-Aware Multi-Modal Video Question Answering (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods for video question answering align visual or textual features directly with large language models, limiting the deep semantic association between modalities and hindering a comprehensive understanding of interactions within spatial and temporal contexts. |
| Approach: | They propose a temporal-aware framework for multi-modal video question answering that aligns videos and questions at fine-grained levels. |
| Outcome: | The proposed framework improves reasoning ability and accuracy of videoQA by aligning videos and questions at fine-grained levels. |
VideoPro: Adaptive Program Reasoning for Long Video Understanding (2026.acl-long)
Copied to clipboard
Chenglin Li, Feng Han, Yikun Wang, Ruilin Li, Shuai Dong, Haowen Hou, Haitao Li, Qianglong Chen, Feng Tao, Jingqi Tong, Yin Zhang, Jiaqi Wang
| Challenge: | Existing methods for understanding long videos are limited due to the sparsity of visual evidence relevant to a given query. |
| Approach: | They propose a framework that enables VideoLLMs to reason over long videos and refine their predictions through executable programs. |
| Outcome: | The proposed framework outperforms existing methods across long-video understanding benchmarks. |
VideoMind: Thinking in Steps for Long Video Understanding (2026.eacl-industry)
Copied to clipboard
Shubhang Bhatnagar, Renxiong Wang, Kapil Krishnakumar, Adel Ahmadyan, Zhaojiang Lin, Lambert Mathias, Xin Luna Dong, Babak Damavandi, Narendra Ahuja, Seungwhan Moon
| Challenge: | Multimodal Large Language Models struggle with Long Video Understanding due to their limited context window and the distributed nature of salient information across many redundant frames. |
| Approach: | They propose a training framework that mimics a human reasoning process to train Long Video Understanding models. |
| Outcome: | The proposed framework achieves 77.6% performance on Video MME, LongVideo, and MLVU benchmarks while yielding 5% improvement on Llama 4 Scout. |
TempCompass: Do Video LLMs Really Understand Videos? (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks on video large language models lack a comprehensive feedback on temporal perception ability . current models cannot distinguish between different temporal aspects and are limited in task formats . |
| Approach: | They propose a benchmark to evaluate temporal perception ability of video large language models . they construct conflicting videos that share the same static content but differ in a specific temporal aspect . |
| Outcome: | The proposed benchmarks show that video large language models exhibit poor temporal perception ability. |
A Video Is Worth 4096 Tokens: Verbalize Videos To Understand Them In Zero Shot (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing annotated training datasets hinder development of supervised learning models for multimedia content . lack of annotating benchmarks hinders development of models with satisfactory performance . a recent study shows that large language models have zero-shot performance in multimedia understanding . |
| Approach: | They propose to verbalize long videos to generate their descriptions in natural language . they then perform video-understanding tasks on the generated story as opposed to the original video . |
| Outcome: | The proposed method achieves better results than baselines for video understanding. |