Papers by Taozhaowen Taozhaowen
Accelerating Dense LLMs via L0-regularized Mixture-of-Experts (2025.acl-short)
Copied to clipboard
| Challenge: | Existing methods for accelerating large language models (LLMs) suffer from slow and costly inference. |
| Approach: | They propose a lightweight MoE approach using cluster confusion matrix and dynamic batching to accelerate dense LLMs. |
| Outcome: | The proposed method achieves 2.5x speedup over dense models while maintaining competitive performance. |