Papers by Elias Frantar
The Optimal BERT Surgeon: Scalable and Accurate Second-Order Pruning for Large Language Models (2022.emnlp-main)
Copied to clipboard
Eldar Kurtic, Daniel Campos, Tuan Nguyen, Elias Frantar, Mark Kurtz, Benjamin Fineran, Michael Goin, Dan Alistarh
| Challenge: | Pre-trained Transformer models provide robust language representations which can be specialized on various tasks. |
| Approach: | They propose an efficient pruning method based on approximate second-order information that allows pruning weight blocks to be used for pruning. |
| Outcome: | The proposed method is the first to be applied at the BERT scale and significantly pushes the boundaries of the current sparse models with respect to all metrics: model size, inference speed and task accuracy. |
QUIK: Towards End-to-end 4-Bit Inference on Generative Large Language Models (2024.emnlp-main)
Copied to clipboard
Saleh Ashkboos, Ilia Markov, Elias Frantar, Tingxuan Zhong, Xincheng Wang, Jie Ren, Torsten Hoefler, Dan Alistarh
| Challenge: | Large Language Models (LLMs) are extremely popular, leading to a race towards reducing their inference costs. |
| Approach: | They propose a method that quantizes weights and activations to 4 bits to achieve better accuracy. |
| Outcome: | The proposed method reduces runtime costs in memory-bound models but does not address cost-bound scenarios. |