Papers by Dan Alistarh
“Give Me BF16 or Give Me Death”? Accuracy-Performance Trade-Offs in LLM Quantization (2025.acl-long)
Copied to clipboard
| Challenge: | despite popularity of large language model quantization, there are significant accuracy-performance trade-offs associated with quantization formats. |
| Approach: | They evaluate popular quantization formats across academic benchmarks and real-world tasks . they also examine the difference in text generated by quantized models versus their uncompressed counterparts . |
| Outcome: | The proposed format is lossless across all model scales and incurs low accuracy degradation when properly tuned. |
The Optimal BERT Surgeon: Scalable and Accurate Second-Order Pruning for Large Language Models (2022.emnlp-main)
Copied to clipboard
Eldar Kurtic, Daniel Campos, Tuan Nguyen, Elias Frantar, Mark Kurtz, Benjamin Fineran, Michael Goin, Dan Alistarh
| Challenge: | Pre-trained Transformer models provide robust language representations which can be specialized on various tasks. |
| Approach: | They propose an efficient pruning method based on approximate second-order information that allows pruning weight blocks to be used for pruning. |
| Outcome: | The proposed method is the first to be applied at the BERT scale and significantly pushes the boundaries of the current sparse models with respect to all metrics: model size, inference speed and task accuracy. |
QUIK: Towards End-to-end 4-Bit Inference on Generative Large Language Models (2024.emnlp-main)
Copied to clipboard
Saleh Ashkboos, Ilia Markov, Elias Frantar, Tingxuan Zhong, Xincheng Wang, Jie Ren, Torsten Hoefler, Dan Alistarh
| Challenge: | Large Language Models (LLMs) are extremely popular, leading to a race towards reducing their inference costs. |
| Approach: | They propose a method that quantizes weights and activations to 4 bits to achieve better accuracy. |
| Outcome: | The proposed method reduces runtime costs in memory-bound models but does not address cost-bound scenarios. |
Mathador-LM: A Dynamic Benchmark for Mathematical Reasoning on Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | a new benchmark for evaluating the mathematical reasoning on large language models is being developed . popularity of reasoning benchmarks is leading to performance saturation and training set contamination. |
| Approach: | They introduce a benchmark for evaluating the mathematical reasoning on large language models . they find that models struggle with Mathador-LM, scoring lower than average 3rd graders . |
| Outcome: | The proposed benchmark improves performance on large language models . it also reduces test-set leakage into training data, a new study shows . |
HIGGS: Pushing the Limits of Large Language Model Quantization via the Linearity Theorem (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for quantizing large language models focus on breaking down the problem into layer-wise sub-problems and minimizing per-layer error, but this approach lacks theoretical justification and the metrics employed may be sub-optimal. |
| Approach: | They propose a "linearity theorem" establishing a direct relationship between the layer-wise reconstruction error and the model perplexity increase due to quantization. |
| Outcome: | The proposed method outperforms previous data-free methods and improves accuracy-compression trade-offs on Llama-family models. |
Speculative Decoding Speed-of-Light: Optimal Lower Bounds via Branching Random Walks (2026.eacl-long)
Copied to clipboard
| Challenge: | Speculative generation has emerged as a promising technique to accelerate inference in large language models (LLMs) however, the fundamental limits on the achievable speedup remain poorly understood. |
| Approach: | They propose to draw a parallel token generation process and branching random walks to achieve the first "tight" lower bounds on the runtime of any deterministic speculative generation algorithm. |
| Outcome: | The proposed method reduces inference latency without altering the output distribution. |