Papers by Gabriele Oliaro
Quantized Side Tuning: Fast and Memory-Efficient Tuning of Quantized Large Language Models (2024.acl-long)
Copied to clipboard
Zhengxin Zhang, Dan Zhao, Xupeng Miao, Gabriele Oliaro, Zhihao Zhang, Qing Li, Yong Jiang, Zhihao Jia
| Challenge: | Existing methods to finetun large language models (LLMs) only update a small number of trainable parameters, or attempt to reduce the memory footprint during the training phase of the finetune process. |
| Approach: | They propose quantized side tuing (QST) which quantizes an LLM’s model weights into 4-bit to reduce the memory footprint of the original weights. |
| Outcome: | The proposed method reduces the memory footprint of the model weights, optimizer states, and intermediate activations while reducing the memory requirements. |