| Challenge: | Language-based learning models (LLMs) support long context-lengths but their effectiveness in handling long-term information gradually declines with input length. |
| Approach: | They propose a Language Repository (LangRepo) that maintains concise and structured information as an interpretable representation. |
| Outcome: | The proposed framework is evaluated on zero-shot visual question-answering benchmarks. |
Similar Papers
Sharper and Faster mean Better: Towards More Efficient Vision-Language Model for Hour-scale Long Video Understanding (2025.acl-long)
Copied to clipboard
| Challenge: | Existing multimodal large language models (LLMs) have shown impressive performance on the video understanding task, but extremely long videos still pose significant challenges to their context length, memory consumption, and computational complexity. |
| Approach: | They propose a vision-language model named Sophia for long video understanding which can efficiently handle hour-scale long videos. |
| Outcome: | The proposed model exhibits competitive performance compared to existing video understanding baselines across various benchmarks for long video understanding with reduced time and memory consumption. |
VideoINSTA: Zero-shot Long Video Understanding via Informative Spatial-Temporal Reasoning with LLMs (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Long video understanding presents unique challenges due to the complexity of reasoning over extended timespans. |
| Approach: | They propose a framework VideoINSTA to leverage large language models for video understanding . they propose 'event-based temporalreasoning' and 'content-based spatial reasoning' |
| Outcome: | The proposed model significantly improves state-of-the-art on three long video question-answering benchmarks. |
TemporalVLM: Video LLMs for Temporal Reasoning in Long Videos (2026.findings-acl)
Copied to clipboard
| Challenge: | Several video understanding applications require the ability of temporal reasoning. |
| Approach: | They propose a video large language model for temporal reasoning and fine-grained understanding in long videos. |
| Outcome: | The proposed model outperforms existing methods in time and motion studies and temporal action segmentation evaluations. |
LiveLongBench: Tackling Long-Context Understanding for Spoken Texts from Live Streams (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing studies show that spoken text exhibits unique linguistic properties, such as high redundancy and repetitive phrases. |
| Approach: | They propose a long-text dataset that better handles redundancy in spoken text . their results highlight key limitations of current methods and suggest future directions . |
| Outcome: | The proposed benchmark improves existing methods and improves on redundancy in spoken text. |
Video-LLaVA: Learning United Visual Representation by Alignment Before Projection (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to visual-language understanding lack unified tokenization for images and videos . lack of unified visual representations makes it difficult to learn multi-modal interactions from poor projection layers. |
| Approach: | They propose to unify visual representation into the language feature space to advance the foundational LLM towards a unified LVLM. |
| Outcome: | The proposed model outperforms Video-ChatGPT on image benchmarks and on 9 image benchmark benchmarks. |
LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding (2024.acl-long)
Copied to clipboard
Yushi Bai, Xin Lv, Jiajie Zhang, Hongchang Lyu, Jiankai Tang, Zhidian Huang, Zhengxiao Du, Xiao Liu, Aohan Zeng, Lei Hou, Yuxiao Dong, Jie Tang, Juanzi Li
| Challenge: | Large language models (LLMs) can only handle texts a few thousand tokens long, limiting their applications on longer sequence inputs, such as books, reports, and codebases. |
| Approach: | They propose a bilingual, multi-task benchmark for long context understanding that extends context windows and more sophisticated memory mechanisms to improve models' long context capabilities. |
| Outcome: | The proposed model outperforms open-source models but struggles on longer contexts. |
ViLL-E: Video LLM Embeddings for Retrieval (2026.acl-long)
Copied to clipboard
| Challenge: | Video Large Language Models excel at video understanding tasks where outputs are textual . however, they underperform specialized embedding-based models in Retrieval tasks . |
| Approach: | They propose a video-LLM-based model with an embedding generation mechanism that allows the model to "think longer" for complex videos and stop early for easy ones. |
| Outcome: | The proposed model outperforms specialized embedding-based models in video understanding tasks while remaining competitive on VideoQA tasks. |
Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | a surge of deep learning applications for video understanding have led to major advancements in video-related tasks. |
| Approach: | They propose a multimodal video-based conversation model that merges a video-adapted visual encoder with an LLM and a dataset that is easily scalable and robust to label noise. |
| Outcome: | The proposed model can understand and generate detailed conversations about videos. |
LLM×MapReduce: Simplified Long-Sequence Processing using Large Language Models (2025.acl-long)
Copied to clipboard
Zihan Zhou, Chong Li, Xinyi Chen, Shuo Wang, Yu Chao, Zhili Li, Haoyu Wang, Qi Shi, Zhixing Tan, Xu Han, Xiaodong Shi, Zhiyuan Liu, Maosong Sun
| Challenge: | Existing studies have focused on extending the context length of large language models (LLMs) due to their quadratic computational complexity and a lack of high-quality long training examples, most LLMs are trained with a limited window size. |
| Approach: | They propose a training-free framework that enables large language models to effectively process long texts using a divide-and-conquer strategy for comprehensive document understanding. |
| Outcome: | The proposed framework outperforms open-source and commercial long-context LLMs and is compatible with several models. |
LongRecipe: Recipe for Efficient Long Context Generalization in Large Language Models (2025.acl-long)
Copied to clipboard
Zhiyuan Hu, Yuliang Liu, Jinman Zhao, Suyuchen Wang, WangYan WangYan, Wei Shen, Qing Gu, Anh Tuan Luu, See-Kiong Ng, Zhiwei Jiang, Bryan Hooi
| Challenge: | Large language models face significant challenges in handling long-context tasks because of their limited effective context window size during pretraining, which restricts their ability to generalize over extended sequences. |
| Approach: | They propose a training strategy for extending the context window of LLMs including impactful token analysis, position index transformation, and training optimization strategies. |
| Outcome: | Experiments on three types of LLMs show that LongRecipe can utilize long sequences while requiring only 30% of the target context window size. |