Papers by Maxim Zhelnin
From 2:4 to 8:16 sparsity patterns in LLMs for Outliers and Weights with Variance Correction (2026.acl-industry)
Copied to clipboard
Egor Maximov, Yulia Kuzkina, Egor Shvetsov, Azamat Kanametov, Aleksandr Prutko, Maxim Zhelnin, Aleksei Goncharov
| Challenge: | Quantization and sparsification are important for large language models, but they struggle to meet performance thresholds due to limited flexibility and sensitivity to outlier weights. |
| Approach: | They propose to use 8:16 semi-structured sparsity to surpass performance thresholds . they also show that structured sparsification for outliers is competitive with unstructured approaches . |
| Outcome: | The proposed method surpasses the Performance Threshold, compared to 2:4 sparsity, and offers greater flexibility with minimal storage overhead. |
GIFT-SW: Gaussian noise Injected Fine-Tuning of Salient Weights for LLMs (2025.acl-long)
Copied to clipboard
Maxim Zhelnin, Viktor Moskvoretskii, Egor Shvetsov, Maria Krylova, Venediktov Egor, Zuev Aleksandr, Evgeny Burnaev
| Challenge: | Recent studies show that a small subset of weights significantly impacts performance. |
| Approach: | They propose a Gaussian noise-injected fine-tuning method that updates only salient weights while injecting Gausssian into non-salient weight. |
| Outcome: | The proposed method outperforms full fine-tuning and PEFT methods under the same computational budget. |
Motivating Next-Gen Accelerators with Flexible N:M Activation Sparsity via Benchmarking Lightweight Post-Training Sparsification Approaches (2026.acl-industry)
Copied to clipboard
Shirin Alanova, Kristina Kazistova, Ekaterina Galaeva, Alina Kostromina, Vladimir Smirnov, Redko Dmitry, Alexey Dontsov, Maxim Zhelnin, Evgeny Burnaev, Egor Shvetsov
| Challenge: | Recent studies show that sparsification is not supported in large language models. |
| Approach: | They propose to use activation pruning to accelerate large language models with sparsification . they compare activation pruners with weight pruner and activater pruning with activation . |
| Outcome: | The proposed approach outperforms weight pruning at matched sparsity levels. |