Papers by Jinheng Wang
Bitnet.cpp: Efficient Edge Inference for Ternary LLMs (2025.acl-long)
Copied to clipboard
Jinheng Wang, Hansong Zhou, Ting Song, Shijie Cao, Yan Xia, Ting Cao, Jianyu Wei, Shuming Ma, Hongyu Wang, Furu Wei
| Challenge: | 1-bit large language models have spurred interest in ternary LLMs, but efficient edge inference is still scarce. |
| Approach: | They propose an inference system optimized for 1-bit large language models . they propose a new library that facilitates sub-2-bits-per-weight inference . |
| Outcome: | The proposed inference system achieves 6.25x speed increase over full-precision baselines and 2.32x over low-bit baselines. |