Papers by Lidan Shou

SkipBERT: Efficient Inference with Shallow Layer Skipping (2022.acl-long)

Copied to clipboard

Jue Wang, Ke Chen, Gang Chen, Lidan Shou, Julian McAuley

Challenge:	Pre-trained language models have significant demands in computation and inference time, limiting their use in resource-constrained or latencysensitive applications.
Approach:	They propose to encode text chunks into independent representations and skip computation of shallow layers to accelerate inference.
Outcome:	The proposed approach can reduce latency by 65% without sacrificing performance.

Efficient Inference for Large Vision-Language Models: Bottlenecks, Techniques, and Prospects (2026.findings-acl)

Copied to clipboard

Jun Zhang, Yicheng Ji, Feiyang Ren, Yihang Li, Bowen Zeng, Zonghao Chen, Ke Chen, Lidan Shou, Gang Chen, Huan Li

Language Models

Challenge:	Large Vision-Language Models are hindered by a systemic efficiency barrier known as visual token dominance.
Approach:	They propose a systematic taxonomy of efficiency techniques structured around the inference lifecycle . they examine visual encoding, prefilling, and decoding to understand bottlenecks .
Outcome:	The proposed techniques reveal how upstream decisions dictate downstream bottlenecks . the proposed techniques include hybrid compression and modality-aware decoding .

Draft & Verify: Lossless Large Language Model Acceleration via Self-Speculative Decoding (2024.acl-long)

Copied to clipboard

Jun Zhang, Jue Wang, Huan Li, Lidan Shou, Ke Chen, Gang Chen, Sharad Mehrotra

Language Models

Challenge:	Existing methods for accelerating Large Language Models have been criticized for their inference costs and inefficient decoding.
Approach:	They propose a self-speculative decoding approach for accelerating Large Language Models without an auxiliary model.
Outcome:	The proposed method achieves a speedup of up to 1.99 with no additional neural network training and no extra memory footprint.

Transfer-Aware Data Selection for Domain Adaptation in Text Retrieval (2025.findings-emnlp)

Copied to clipboard

Linzhu Yu, Huan Li, Ke Chen, Lidan Shou

Semantic Text Processing

Challenge:	Existing methods to improve domain adaptation do not guarantee improved adaptability, but may negatively impact model performance.
Approach:	They propose a framework that can effectively improve model adaptability by selecting beneficial data without evaluating all source data.
Outcome:	The proposed framework improves model adaptability by selecting beneficial data without evaluating all source data.

T2DR: A Two-Tier Deficiency-Resistant Framework for Incomplete Multimodal Learning (2025.findings-acl)

Copied to clipboard

Han Lin, Xiu Tang, Huan Li, Wenxue Cao, Sai Wu, Chang Yao, Lidan Shou, Gang Chen

Multimodality

Challenge:	Existing incomplete multimodal learning frameworks are inadequate for integrating multimodal data.
Approach:	They propose a framework for incomplete multimodal learning that is deficiency-resistant and provides two modules to address fine-grained deficiencies.
Outcome:	The proposed framework outperforms the SOTA models on two well-known multimodal benchmarks.

MM-ShiftKV: Decode-Aware Prefill-Stage KV Selection for Multimodal Large Language Models (2026.findings-acl)

Copied to clipboard

Jinsong Shu, Chenyang Wu, Zhongle Xie, Baokun Wang, Lidan Shou

Multimodality

Challenge:	Recent work suggests a prefill-stage KV cache selection method to estimate KV importance from prefilling statistics.
Approach:	They propose a training-free, decode-aware and strictly prefill-only KV selection method that retains key-value caching for decoding .
Outcome:	The proposed method outperforms existing methods under tight cache budgets on multimodal benchmarks.

HybridKV: Hybrid KV Cache Compression for Efficient Multimodal Large Language Model Inference (2026.acl-long)

Copied to clipboard

Bowen Zeng, Feiyang Ren, Jun Zhang, Xiaoling Gu, Ke Chen, Lidan Shou, Huan Li

Semantic Text Processing

Challenge:	Multimodal Large Language Models (MLLMs) are hindered by the rapid growth of key–value (KV) caches.
Approach:	They propose a hybrid KV cache compression framework that reduces KV memory by 7.9 and speeds up decoding by 1.52.
Outcome:	Experiments on 11 multimodal benchmarks show that HYBRIDKV cuts KV cache memory by 7.9 and speeds up decoding by 1.52.

SpecVLM: Enhancing Speculative Decoding of Video LLMs via Verifier-Guided Token Pruning (2025.emnlp-main)

Copied to clipboard

Yicheng Ji, Jun Zhang, Heming Xia, Jinpeng Chen, Lidan Shou, Gang Chen, Huan Li

Multimodality

Challenge:	Video large language models (Vid-LLMs) rely on dense video token representations and require substantial memory and computational overhead in both prefilling and decoding.
Approach:	They propose a training-free speculative decoding framework that prunes up to 90% of video tokens to enable efficient speculation without sacrificing accuracy.
Outcome:	The proposed framework achieves 2.68 speedup on LLaVA-OneVision-72B and 2.11 speed up on Qwen2.5-VL-32B.

TokenTiming: A Dynamic Alignment Method for Universal Speculative Decoding Model Pairs (2026.acl-long)

Copied to clipboard

Sibo Xiao, Fu Jinyuan, Zhongle Xie, Lidan Shou

Challenge:	Speculative decoding (SD) is a useful tool for accelerating large language models . but its utility is limited by a fundamental constraint: draft and target models must share the same vocabulary .
Approach:	They propose an algorithm that uses a draft token sequence to get a new target token sequence and then uses DTW to build a mapping to transfer probability distributions.
Outcome:	The proposed method shows 1.57x speedup on various tasks.

Pyramid: A Layered Model for Nested Named Entity Recognition (2020.acl-main)

Copied to clipboard

Jue Wang, Lidan Shou, Ke Chen, Gang Chen

Semantic Text Processing

Challenge:	Named Entity Recognition (NER) is a fundamental NLP task.
Approach:	They propose a pyramid-like layered model for Nested Named Entity Recognition . token or text region embeddings are recursively inputted into L flat NER layers .
Outcome:	The proposed model achieves state-of-the-art F1 scores in nested NER on ACE-2004, ACE 2005, GENIA, and NNE.

See the Forest for the Trees: Loosely Speculative Decoding via Visual-Semantic Guidance for Efficient Inference of Video LLMs (2026.acl-long)

Copied to clipboard

Yicheng Ji, Jun Zhang, Jinpeng Chen, Cong Wang, Lidan Shou, Gang Chen, Huan Li

Multimodality

Challenge:	Existing methods for video understanding suffer from autoregressive generation of tokens.
Approach:	They propose a training-free loosely SD framework for Video-LLMs that uses visual-relevant tokens to accurately pinpoint the latter.
Outcome:	The proposed framework boosts the accepted length and speedup ratio by 136% and 35% compared to SOTA training-free SD methods for Video-LLMs.