Papers by Sourav Bhattacharya
MobileQuant: Mobile-friendly Quantization for On-device Language Models (2024.findings-emnlp)
Copied to clipboard
Fuwen Tan, Royson Lee, Łukasz Dudziak, Shell Xu Hu, Sourav Bhattacharya, Timothy Hospedales, Georgios Tzimiropoulos, Brais Martinez
| Challenge: | Large language models (LLMs) have revolutionized language processing, but deployment on edge devices is costly in terms of memory, computation and energy. |
| Approach: | They propose to reduce the number of bits used to represent weights and activations . they propose to use 8-bit activations to enable LLMs to fully exploit mobile-friendly hardware . |
| Outcome: | The proposed method reduces the number of bits used to represent weights and activations . 8-bit activations are attractive for on-device deployment as they would exploit mobile-friendly hardware . |