Profiling-Free Mixed-Precision Quantization for MoE LLMs via Fuzzy Rule Interpolation (2026.acl-long)
Copied to clipboard
| Challenge: | Large Language Models are scaling in size and capability, driving substantial computational and memory costs. |
| Approach: | They propose a mixed-precision quantization framework that uses fuzzy rule interpolation to predict quantization error from only sparse samples. |
| Outcome: | The proposed framework accelerates the profiling phase by up to 15.7 on DeepSeek-V2 while achieving comparable or slightly superior zero-shot accuracy. |
Similar Papers
Automated Fine-Grained Mixture-of-Experts Quantization (2025.findings-acl)
Copied to clipboard
| Challenge: | specialized quantization framework for Mixture of Experts architectures is inadequate for model compression. |
| Approach: | They propose a specialized quantization framework for Mixture of Experts architectures . they find that expert networks exhibit distinctive channel-wise outlier distributions ." |
| Outcome: | The proposed framework improves on the Mixtral-8x7b-v0.1 architecture while maintaining minimal computational overhead. |
MoQAE: Mixed-Precision Quantization for Long-Context LLM Inference via Mixture of Quantization-Aware Experts (2025.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to optimize large language models for long-context inference are inefficient and consume memory. |
| Approach: | They propose a mixed-precision quantization method via mixture of experts that inputs tokens into router chunk by chunk to reduce inference overhead. |
| Outcome: | The proposed method outperforms state-of-the-art KV cache quantization methods on multiple benchmark datasets. |
EAC-MoE: Expert-Selection Aware Compressor for Mixture-of-Experts Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Mixture-of-Experts (MoE) has demonstrated promising potential in scaling LLMs . however, it is hindered by two critical challenges: substantial GPU memory consumption and low activated parameters. |
| Approach: | They propose an Expert-Selection Aware Compressor for Mixture-of-Experts (MoE) that aligns with the characteristics of MoE from the perspectives of quantization and pruning. |
| Outcome: | The proposed approach significantly reduces memory usage and improves inference speed with minimal performance degradation. |
Faster MoE LLM Inference for Extremely Large Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing inference optimizations for coarse-grained Mixture-of-Experts models implicitly assume a fixed activation budget, which is poorly understood. |
| Approach: | They propose a training-free policy that adapts token-level activation using router confidence and entropy while remaining within the model’s original budget. |
| Outcome: | The proposed skipping policy can provide substantial throughput gains, but optimal static schedules vary significantly across models and routing mechanisms. |
AMQ: Enabling AutoML for Mixed-precision Weight-Only Quantization of Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Weight-only quantization is a powerful optimization technique for large language models . pushing below 4 bits often leads to substantial accuracy degradation due to increased quantization error. |
| Approach: | They propose a framework that assigns layer-wise quantization bit-widths to optimize model quality and memory usage. |
| Outcome: | The proposed framework can optimize for large language models under memory constraints. |
XMoE: Sparse Models with Fine-grained and Adaptive Expert Selection (2024.findings-acl)
Copied to clipboard
| Challenge: | XMoE leverages small experts and a threshold-based router to selectively engage only essential parameters. |
| Approach: | They propose a novel MoE that leverages small experts to selectively engage only essential parameters. |
| Outcome: | The proposed model can reduce computation load at MoE layers by over 50% without sacrificing performance. |
MoE-I2: Compressing Mixture of Experts Models through Inter-Expert Pruning and Intra-Expert Low-Rank Decomposition (2024.findings-emnlp)
Copied to clipboard
Cheng Yang, Yang Sui, Jinqi Xiao, Lingyi Huang, Yu Gong, Yuanlin Duan, Wenqi Jia, Miao Yin, Yu Cheng, Bo Yuan
| Challenge: | emergence of Mixture of Experts (MoE) LLMs has significantly advanced the development of language models. |
| Approach: | They propose a two-stage compression method tailored for Mixture of Experts to reduce the model size and decrease the computational cost. |
| Outcome: | The proposed method reduces model size and improves inference efficiency while maintaining performance in various zero-shot tasks. |
Balancing Fidelity and Plasticity: Aligning Mixed-Precision Fine-Tuning with Linguistic Hierarchies (2026.findings-acl)
Copied to clipboard
Changhai Zhou, Shiyang Zhang, Yuhua Zhou, Jun Gao, Qian Qiao, Shichao Weng, Weizhong Zhang, Cheng Jin
| Challenge: | Existing quantization-aware fine-tuning methods decouple weight precision and adapter capacity, overlooking that a layer’s ability to adapt is constrained by the information preserved in its frozen weights. |
| Approach: | They propose a framework that jointly optimizes per-layer quantization bit-width and LoRA rank. |
| Outcome: | Experiments on LLaMA and Qwen models show that the proposed framework matches or approaches 16-bit baselines while using substantially less memory. |
Decoding Knowledge Attribution in Mixture-of-Experts: A Framework of Basic-Refinement Collaboration and Efficiency Analysis (2025.acl-long)
Copied to clipboard
| Challenge: | Existing attribution methods for dense models fail to capture dynamic routing-expert interactions in sparse MoE architectures. |
| Approach: | They propose to analyze sparse MoE architectures against dense models to capture dynamic routing-expert interactions. |
| Outcome: | The proposed algorithm shows that sparse models achieve higher efficiency per layer . it also shows that deep Qwen-MoE mitigates expert failures while minimizing complexity . |
Bag of Tricks for Sparse Mixture-of-Experts: A Benchmark Across Reasoning, Efficiency, and Safety (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing benchmarks focus on isolated aspects of MoE, with conflicting conclusions . a lack of consensus on optimal design choices is limiting to specific aspects of the model. |
| Approach: | They propose to evaluate two popular MoE backbones across four dimensions of design choices . they find token-level routing and z-loss regularization improve reasoning performance . |
| Outcome: | The proposed framework evaluates two popular MoE backbones on over eight metrics. |