SmartTrim: Adaptive Tokens and Attention Pruning for Efficient Vision-Language Models (2024.lrec-main)
Copied to clipboard
Zekun Wang, Jingchang Chen, Wangchunshu Zhou, Haichao Zhu, Jiafeng Liang, Liping Shan, Ming Liu, Dongliang Xu, Qing Yang, Bing Qin
| Challenge: | Experimental results show that SmartTrim accelerates the original model by 2-3 times with minimal performance degradation. |
| Approach: | They propose an adaptive acceleration framework which prunes redundant token representations and attention heads within each layer of the original model. |
| Outcome: | The proposed framework accelerates the original model by 2-3 times with minimal performance degradation across vision-language tasks. |
Similar Papers
AdaV: Adaptive Text-visual Redirection for Vision-Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Vision-language models often generate excessive visual tokens, leading to poor performance . a novel training-free visual token pruning method is proposed to improve performance despite the computational cost associated with VLMs. |
| Approach: | They propose a training-free visual token pruning method that reduces biased token pruning . they plan to open-source the code upon publication . |
| Outcome: | The proposed method reduces biased token pruning and enhances model robustness with limited visual token budget. |
HiPrune: Hierarchical Attention for Efficient Token Pruning in Vision-Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for visual token pruning lack insight into the intrinsic property of the vision encoder . et al., 2017: 99.3% of task accuracy with only 1/3 of the tokens. |
| Approach: | They propose a model-agnostic token pruning method that trains without training . they propose 'HiPrune' method which prunes visual tokens according to their attention . |
| Outcome: | The proposed method achieves 99.3% of task accuracy with only 1/3 of the tokens . it reduces inference FLOPs by 58.7% and maintains 99.99% accuracy with 2/9 tokens. |
EfficientVLM: Fast and Accurate Vision-Language Models via Knowledge Distillation and Modal-adaptive Pruning (2023.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained vision-language models have achieved impressive results in a range of vision-linguistic tasks. |
| Approach: | They propose a distilling then pruning framework to compress large vision-language models into smaller, faster ones. |
| Outcome: | The proposed framework reduces the size of a pre-trained large vision-language model and improves its performance on vision-linguistic tasks. |
Reducing Token Redundancy in LVLMs: A Systematic Review of Token Pruning Methods (2026.acl-long)
Copied to clipboard
| Challenge: | Large Vision-Language Models (LVLMs) excel at visual understanding but face severe computational bottlenecks when processing high-resolution images and long videos due to massive visual token counts. |
| Approach: | They propose a taxonomy categorizing methods into vision-side, LLM-side and hybrid paradigms and analyze token selection mechanisms and pruning strategy. |
| Outcome: | The proposed method selectively removes less informative tokens while maintaining performance. |
VisPCO: Visual Token Pruning Configuration Optimization via Budget-Aware Pareto-Frontier Learning for Vision-Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for visual token pruning rely on predefined configurations without determining whether they achieve optimal performance. |
| Approach: | They propose a framework that formulates visual token pruning as a Pareto configuration optimization problem to automatically identify optimal configurations. |
| Outcome: | The proposed framework approximates the empirical Pareto frontier obtained through grid search and generalizes well across pruning methods and VLM architectures. |
LVPruning: An Effective yet Simple Language-Guided Vision Token Pruning Approach for Multi-modal Large Language Models (2025.findings-naacl)
Copied to clipboard
| Challenge: | Multi-modal Large Language Models (MLLMs) incur significant computational overhead due to the large number of vision tokens processed, limiting their practicality in resource-constrained environments. |
| Approach: | They propose a language-guided vision token pruning method that can be integrated into existing MLLMs with minimal architectural changes. |
| Outcome: | The proposed method reduces vision tokens by 90% and preserves model performance. |
PruneVid: Visual Token Pruning for Efficient Video Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to video token pruning face significant computational challenges due to the redundancy inherent in video data. |
| Approach: | They propose a training-free visual token pruning method that reduces the redundancy inherent in video data and leverages LLMs’ inherent ability to selectively prune visual tokens irrelevant to specific queries. |
| Outcome: | The proposed method can prune over 80% of tokens while maintaining competitive performance when combined with different video LLMs. |
CoViPAL: Layer-wise Contextualized Visual Token Pruning for Large Vision-Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to prune redundant vision tokens struggle in shallow layers due to the lack of contextual information. |
| Approach: | They propose a layer-wise contextualized visual token pruning method that uses a plug-and-play Pruning Module to prune redundant vision tokens. |
| Outcome: | The proposed method outperforms training-free pruning methods under equal token budgets and surpasses training based methods with comparable supervision. |
Focus on the Core: Efficient Attention via Pruned Token Compression for Document Classification (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained transformers suffer from a computationally expensive self-attention mechanism that interacts with all tokens, including those unfavorable to classification performance. |
| Approach: | They propose to integrate token pruning and token combining strategies to improve model performance and reduce computational demands. |
| Outcome: | Experiments with various datasets show that the proposed model performs better than baseline models, with the best improvement over the existing model. |
Token Pruning in Multimodal Large Language Models: Are We Solving the Right Problem? (2025.findings-acl)
Copied to clipboard
| Challenge: | Multimodal large language models have shown remarkable performance for cross-modal understanding and generation, yet suffer from severe inference costs. |
| Approach: | They propose to prune redundant tokens in MLLMs to reduce computation and storage costs. |
| Outcome: | The proposed method reduces the computational and storage costs of MLLMs by identifying redundant tokens and pruning them. |