Papers with NPUs
Unleashing Low-Bit Inference on Ascend NPUs: A Comprehensive Evaluation of HiFloat Formats (2026.acl-industry)
Copied to clipboard
Pengxiang Zhao, Hui-Ling Zhen, Xing Li, Han Bao, Weizhe Lin, Zhiyuan Yang, Yu Zi Wei, Xin Wang, Mingxuan Yuan, Xianzhi Yu, Zhenhua Dong
| Challenge: | Low-bit floating-point formats like MXFP and NVFP4 offer new opportunities for precision and efficiency. |
| Approach: | They evaluate HiFloat (HiF8 and HiF4), a family of floating-point formats tailored for Ascend NPUs. |
| Outcome: | The proposed formats excel with high-variance data and are compatible with state-of-the-art quantization frameworks. |
MobileQuant: Mobile-friendly Quantization for On-device Language Models (2024.findings-emnlp)
Copied to clipboard
Fuwen Tan, Royson Lee, Łukasz Dudziak, Shell Xu Hu, Sourav Bhattacharya, Timothy Hospedales, Georgios Tzimiropoulos, Brais Martinez
| Challenge: | Large language models (LLMs) have revolutionized language processing, but deployment on edge devices is costly in terms of memory, computation and energy. |
| Approach: | They propose to reduce the number of bits used to represent weights and activations . they propose to use 8-bit activations to enable LLMs to fully exploit mobile-friendly hardware . |
| Outcome: | The proposed method reduces the number of bits used to represent weights and activations . 8-bit activations are attractive for on-device deployment as they would exploit mobile-friendly hardware . |
AscendKernelGen: LLM-Driven Kernel Generation for NPUs (2026.findings-acl)
Copied to clipboard
Xinzi Cao, Jianyang Zhai, Pengfei Li, Zhiheng Hu, Cen Yan, null Mubingxu, Guanghuan Fang, Bin She, Jiayu Li, Yihan Su, Dongyang Tao, Feidiao Yang, Chang-Dong Wang, Yutong Lu, Weicheng Xue, Bin Zhou, Yonghong Tian
| Challenge: | Neural Processing Units (NPUs) are critical for AI infrastructure, but their development remains a bottleneck due to vendor-specific Domain-Specific Languages (DSLs). |
| Approach: | They propose a framework for NPU kernel development that bridges the gap in hardware-specific coding . compiler success on complex Level-2 kernels improves from 0% to 95.5%, they say . |
| Outcome: | The proposed framework bridges the gap in hardware-specific coding, showing a near-zero success rate on complex kernels. |