Papers with acceleration
MediSwift: Efficient Sparse Pre-trained Biomedical Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models are typically trained on general source data forvarious domains, but domain-specific pre-training is expensive and requires computational costs. |
| Approach: | They propose a suite of biomedicalLMs that leverage sparse pre-training on domain-specific biomedically text data. |
| Outcome: | The proposed model outperforms existing LLMs on biomedical tasks by 22.5x . |
Boost Transformer-based Language Models with GPU-Friendly Sparsity and Quantization (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing studies have shown that transformer-based language models can achieve state-of-the-art performance on various natural language processing tasks. |
| Approach: | They propose a GPU-friendly 2:4 fine-grained structured sparsity and quantization scheme to maximize the GPU's acceleration constraints. |
| Outcome: | The proposed scheme achieves state-of-the-art compression on TLM model of various encoder and decoder blocks with negligible accuracy degradation on SQuAD, GLUE, CNN-DM & XSum and WikiText benchmarking tasks. |
KNN-SSD: Enabling Dynamic Self-Speculative Decoding via Nearest Neighbor Layer Set Optimization (2026.findings-eacl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have proven highly capable in handling downstream tasks, but the token-by-token generation in autoregressive decoding results in quadratic computational complexity. |
| Approach: | They propose a method that proposes skipping certain layers to construct a draft model, which eliminates the need for additional parameters or training. |
| Outcome: | The proposed method achieves 1.31.6 speedup in LLM inference while being sensitive to domain shifts. |
ProSparse: Introducing and Enhancing Intrinsic Activation Sparsity within Large Language Models (2025.coling-main)
Copied to clipboard
Chenyang Song, Xu Han, Zhengyan Zhang, Shengding Hu, Xiyu Shi, Kuai Li, Chen Chen, Zhiyuan Liu, Guangli Li, Tao Yang, Maosong Sun
| Challenge: | Activation sparsity is a promising paradigm for accelerating model inference . few large language models achieve high activation spar and comparable performance . |
| Approach: | They propose a method to achieve activation sparsity and acceleration in large language models . they introduce ReLU activation and adopt progressive sparse regularization . |
| Outcome: | The proposed method achieves high activation sparsity and comparable model performance. |
GKT: A Novel Guidance-Based Knowledge Transfer Framework For Efficient Cloud-edge Collaboration LLM Deployment (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods of acceleration require fine-tuning of considerably large models, such as Llama-7B, posing a challenge for average users. |
| Approach: | They propose a Guidance-based Knowledge Transfer framework that leverages a larger LLM as a 'teacher' and a smaller 'student' model to finalize responses. |
| Outcome: | The proposed framework achieves a maximum accuracy improvement of 14.18%, along with a 10.72 times speed-up on GSM8K and an accuracy improvement 14.00% along with 7.73 times speed up in CSQA. |
An Empirical Study of Speculative Decoding for Small Language Models (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing studies focus on 7B-70B parameters models, leaving a knowledge gap for small language models. |
| Approach: | They propose a draft-then-verify paradigm that allows for a single forward pass through a model and transfer of all model parameters to the GPU cache. |
| Outcome: | The proposed method can be used to accelerate small language models with low computational overhead. |
Does Acceleration Cause Hidden Instability in Vision Language Models? Uncovering Instance-Level Divergence Through a Large-Scale Empirical Study (2025.emnlp-main)
Copied to clipboard
| Challenge: | Current acceleration evaluations focus on minimal overall performance degradation . however, accelerated models can exhibit significant changes in instance-level predictions . |
| Approach: | They investigate whether accelerated vision-Language Models can still give the same answers as before . they found that accelerated models changed original answers up to 20% of the time . |
| Outcome: | The results show that accelerated models changed their original answers up to 20% of the time. |
Faster Speculative Decoding via Effective Draft Decoder with Pruned Candidate Tree (2025.acl-long)
Copied to clipboard
| Challenge: | Effective Draft Decoder (EDD) is a powerful decoding method that generates more accurate draft tokens by leveraging the encoding results as soft prompts. |
| Approach: | They propose an effective draft decoder which treats the LLM as a powerful encoder and generates more accurate draft tokens by leveraging the encoding results as soft prompts. |
| Outcome: | The proposed method significantly improves the performance of large language models and reduces inference latency. |
Towards Efficient and Effective Diffusion Language Model Inference via Semantic-Aware Adaptive Denoising (2026.acl-long)
Copied to clipboard
| Challenge: | Existing acceleration works cannot accurately detect semantically stabilized tokens and then skip computation, leading to sub-optimal speedup in practice. |
| Approach: | They propose a semantic-aware adaptive denoising framework that encodes scalar confidence scores into an evolution-awful feature vector and clusters vectors proactively and adaptively identify semantically converged tokens. |
| Outcome: | The proposed framework outperforms the SOTA competitor in speed and quality . it can detect semantically stabilized tokens and skip computation, resulting in sub-optimal speedup . |
BWLA: Breaking the Barrier of W1AX Post-Training Quantization for LLMs (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models have driven major progress in NLP, but memory and compute requirements hinder practical deployment. |
| Approach: | They propose a framework that preserves high accuracy while achieving 1-bit weight quantization . the orthogonal-kronecker transformation learns an orthogonale mapping via EM minimization - a new approach to quantization is proposed . |
| Outcome: | The proposed framework achieves 1-bit weight quantization with low activations with low-bit activations. |
SPIDE: Serial and Parallel Intertwined Speculative Decoding (2026.findings-acl)
Copied to clipboard
Wenru Xu, Peixuan Xu, Ziqi Yang, Ming Hu, Zihui Wang, Jianzhong Qi, Rongshan Yu, Xiaoliang Fan, Cheng Wang
| Challenge: | Speculative decoding (SD) is a training-free SD framework that orchestrates dynamic alternation combining serial dynamic drafting with parallel draft verification. |
| Approach: | They propose a serial and parallel intertwined speculative DEcoding framework that orchestrates dynamic alternation combining serial dynamic drafting and parallel draft verification. |
| Outcome: | The proposed framework accelerates inference while reducing the LLM usage costs. |
FastKV: Decoupling of Context Reduction and KV Cache Compression for Prefill-Decoding Acceleration (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) excel at handling long-context sequences, but require substantial prefill computation and key-value (KV) cache. |
| Approach: | They propose a KV cache compression framework that decouples prefill computation from decoding KV budget. |
| Outcome: | The proposed framework reduces latency in prefill and decoding by leveraging the stabilization of token importance in later layers. |