Papers by Mark Kurtz
“Give Me BF16 or Give Me Death”? Accuracy-Performance Trade-Offs in LLM Quantization (2025.acl-long)
Copied to clipboard
| Challenge: | despite popularity of large language model quantization, there are significant accuracy-performance trade-offs associated with quantization formats. |
| Approach: | They evaluate popular quantization formats across academic benchmarks and real-world tasks . they also examine the difference in text generated by quantized models versus their uncompressed counterparts . |
| Outcome: | The proposed format is lossless across all model scales and incurs low accuracy degradation when properly tuned. |
The Optimal BERT Surgeon: Scalable and Accurate Second-Order Pruning for Large Language Models (2022.emnlp-main)
Copied to clipboard
Eldar Kurtic, Daniel Campos, Tuan Nguyen, Elias Frantar, Mark Kurtz, Benjamin Fineran, Michael Goin, Dan Alistarh
| Challenge: | Pre-trained Transformer models provide robust language representations which can be specialized on various tasks. |
| Approach: | They propose an efficient pruning method based on approximate second-order information that allows pruning weight blocks to be used for pruning. |
| Outcome: | The proposed method is the first to be applied at the BERT scale and significantly pushes the boundaries of the current sparse models with respect to all metrics: model size, inference speed and task accuracy. |