Papers by Qingan Li
POP: Prefill-Only Pruning for Efficient Large Model Inference (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing structured pruning methods suffer from significant accuracy degradation . Existing pruning methods are expensive and require specialized hardware and kernels to perform . |
| Approach: | They propose a stage-agnostic pruning approach that overlooks asymmetric roles between prefill and decode stages. |
| Outcome: | The proposed pruning approach achieves 1.37 speedup in prefill latency with minimal performance loss. |
CHESS: Optimizing LLM Inference via Channel-Wise Thresholding and Selective Sparsification (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for activation sparsification do not capture the relationship between activation and model performance. |
| Approach: | They propose a general activation sparsification approach using channel-wise thresholding and selective sparsifying to capture the relationship between activation and model performance. |
| Outcome: | The proposed approach reduces the number of activated neurons during inference by 1.27x over eight downstream tasks while activating fewer parameters than existing methods. |
A2ATS: Retrieval-Based KV Cache Reduction via Windowed Rotary Position Embedding and Query-Aware Vector Quantization (2025.findings-acl)
Copied to clipboard
Junhui He, Junna Xing, Nan Wang, Rui Xu, Shangyu Wu, Peng Zhou, Qiang Liu, Chun Jason Xue, Qingan Li
| Challenge: | Long context large language models (LLMs) pose significant challenges for efficient serving due to the large memory footprint and high access overhead of KV cache. |
| Approach: | They propose a retrieval-based method to reduce the memory footprint of LLMs . they propose Windowed Rotary Position Embedding and query-aware vector quantization . |
| Outcome: | The proposed method can achieve lower performance degradation with lower overhead compared to existing methods . it can reduce the memory footprint and access overhead of long context large language models . |
MLWQ: Efficient Small Language Model Deployment via Multi-Level Weight Quantization (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for efficient deployment of small language models face inefficient bit-width allocation and insufficient fine-grained quantization adjustments. |
| Approach: | They propose a weight quantization technique that facilitates efficient deployment of SLMs . they propose to combine inter-layer loss and intra-layer salience to achieve better allocation . |
| Outcome: | Experimental results show that multi-level weight quantization achieves competitive performance compared to state-of-the-art methods. |