In Defense of Structural Sparse Adapters for Concurrent LLM Serving (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) require adapters to fine tune performance without extensive retraining. |
| Approach: | They propose a system that uses structurally sparse adapters to serve LLMs with multiple structurally-sparse axons. |
| Outcome: | The proposed system achieves 2.12 speedup over low-rank adapters on 96 adapters with a single GPU. |
Similar Papers
1+1>2: A Synergistic Sparse and Low-Rank Compression Method for Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Low-rank approximation compresses the model by retaining its essential structure with minimal information loss. |
| Approach: | They propose a method that leverages the strengths of pruning and low-rank approximation for LLMs. |
| Outcome: | The proposed methods surpass the existing methods on LLaMA and Qwen2.5 models. |
Sparser Mixture-of-Adapters with Cross-Layer Generalization (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for training large language models do not allow sharing adapters across layers . existing methods do not support sharing adapter pools, leading to redundancy and poor generalization . |
| Approach: | They propose a mixture-of-adapter framework that trains a pool of lightweight adapters at each layer and selects the most suitable ones for each input. |
| Outcome: | The proposed framework reduces active adapters by over 85% while boosting task accuracy. |
Sparse Brains are Also Adaptive Brains: Cognitive-Load-Aware Dynamic Activation for LLMs (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing sparsity methods lack adaptivity to contextual or model structural demands or incur prohibitive computational overhead. |
| Approach: | They propose a Cognitive-Load-Aware Dynamic Activation framework that synergizes statistical sparsity with semantic adaptability. |
| Outcome: | The proposed framework achieves 20% average speedup with less than 2% accuracy degradation outperforming Griffin and TT. |
Motivating Next-Gen Accelerators with Flexible N:M Activation Sparsity via Benchmarking Lightweight Post-Training Sparsification Approaches (2026.acl-industry)
Copied to clipboard
Shirin Alanova, Kristina Kazistova, Ekaterina Galaeva, Alina Kostromina, Vladimir Smirnov, Redko Dmitry, Alexey Dontsov, Maxim Zhelnin, Evgeny Burnaev, Egor Shvetsov
| Challenge: | Recent studies show that sparsification is not supported in large language models. |
| Approach: | They propose to use activation pruning to accelerate large language models with sparsification . they compare activation pruners with weight pruner and activater pruning with activation . |
| Outcome: | The proposed approach outperforms weight pruning at matched sparsity levels. |
LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Models (2023.emnlp-main)
Copied to clipboard
Zhiqiang Hu, Lei Wang, Yihuai Lan, Wanyu Xu, Ee-Peng Lim, Lidong Bing, Xing Xu, Soujanya Poria, Roy Lee
| Challenge: | Large language models (LLMs) have shown unprecedented performance across various tasks. |
| Approach: | They propose an easy-to-use framework that integrates adapters into LLMs . they evaluate adapters on 14 datasets from two different reasoning tasks . |
| Outcome: | The proposed framework can be used to fine-tune open-access language models with task-specific data and instruction data. |
SEMIROUTER: Sparse-Data Enhanced Routing for Adaptive Multi-LLM System (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing routing methods suffer from poor scalability and dependence on datasets for training . energy footprint is also considered in the decision to implement our new LLM routing framework . |
| Approach: | They propose a new LLM routing framework that dynamically allocates queries to the most appropriate LLM. |
| Outcome: | The proposed method improves data efficiency, adaptability, and routing accuracy compared to existing methods. |
Hardware-Aware Parallel Prompt Decoding for Memory-Efficient Acceleration of LLM Inference (2025.findings-emnlp)
Copied to clipboard
Hao Mark Chen, Wayne Luk, Yiu Ka Fai Cedric, Rui Li, Konstantin Mishchenko, Stylianos Venieris, Hongxiang Fan
| Challenge: | Auto-regressive decoding of Large Language Models results in significant overheads in hardware performance . a novel parallel prompt decoding approach is proposed to overcome these limitations . |
| Approach: | They propose a parallel prompt decoding that uses a single model for speculation and verification. |
| Outcome: | The proposed approach speeds up auto-regressive decoding of large language models 2.49 times . it can be used on mobileLlama to Vicuna-13B on a wide range of benchmarks . |
Sparse-to-Dense: A Free Lunch for Lossless Acceleration of Video Understanding in LLMs (2025.acl-short)
Copied to clipboard
| Challenge: | Recent advances in Video Large Language Models (Video-LLMs) have achieved exceptional performance on tasks like video question answering and captioning. |
| Approach: | They propose a decoding strategy that leverages sparse top-K attention and dense full attention to accelerate Video-LLMs without loss. |
| Outcome: | The proposed approach achieves a 1.94 walltime speedup in video processing. |
LLMs Are Zero-Shot Context-Aware Simultaneous Translators (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing SiMT systems operate on a sentence level, disregarding the context established by previous sentences or the broader context implied by previous words. |
| Approach: | They show that open-source LLMs perform on par with or better than some state-of-the-art baselines in simultaneous machine translation tasks, zero-shot. |
| Outcome: | The proposed models perform on par with or better than state-of-the-art baselines in simultaneous machine translation tasks, zero-shot. |
LLM in a flash: Efficient Large Language Model Inference with Limited Memory (2024.acl-long)
Copied to clipboard
Keivan Alizadeh, Seyed Iman Mirzadeh, Dmitry Belenko, S. Khatamifard, Minsik Cho, Carlo C Del Mundo, Mohammad Rastegari, Mehrdad Farajtabar
| Challenge: | Large language models (LLMs) have high computational and memory requirements, especially for devices with limited memory. |
| Approach: | They propose a method that stores model parameters in flash memory but brings them on demand to DRAM . authors propose two techniques to optimize for reading data in larger, more contiguous chunks . |
| Outcome: | The proposed method reduces the volume of data transferred from flash and reads data in larger, more contiguous chunks. |