Papers with high-efficiency
Unleashing Low-Bit Inference on Ascend NPUs: A Comprehensive Evaluation of HiFloat Formats (2026.acl-industry)
Copied to clipboard
Pengxiang Zhao, Hui-Ling Zhen, Xing Li, Han Bao, Weizhe Lin, Zhiyuan Yang, Yu Zi Wei, Xin Wang, Mingxuan Yuan, Xianzhi Yu, Zhenhua Dong
| Challenge: | Low-bit floating-point formats like MXFP and NVFP4 offer new opportunities for precision and efficiency. |
| Approach: | They evaluate HiFloat (HiF8 and HiF4), a family of floating-point formats tailored for Ascend NPUs. |
| Outcome: | The proposed formats excel with high-variance data and are compatible with state-of-the-art quantization frameworks. |
SCVQ: Sparse-Compensated Vector Quantization for Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing vector quantization methods incur inference overhead due to massive codebook storage and intensive index lookups. |
| Approach: | They propose a framework for vector quantization that incorporates a salience-aware weighted K-means clustering scheme with symmetry constraints to reduce codebook size and indexing costs. |
| Outcome: | The proposed framework achieves a perplexity of 5.78 on WikiText-2 for LLaMA-2-7B at 2-bit quantization while delivering a 1.4 speedup over existing baselines. |