Papers by Nikita Balagansky
Steering LLM Reasoning Through Bias-Only Adaptation (2025.emnlp-main)
Copied to clipboard
Viacheslav Sinii, Alexey Gorbatovski, Artem Cherepanov, Boris Shaposhnikov, Nikita Balagansky, Daniil Gavrilov
| Challenge: | Compared with LoRA and BitFit, training a single steering vector per layer with reinforcement learning requires orders of magnitude fewer resources and isolates a much smaller, more interpretable parameter set. |
| Approach: | They propose to train a single steering vector per layer with reinforcement learning while freezing all base weights to match the accuracy of fully RL-tuned reasoning models. |
| Outcome: | The proposed approach improves on an 8 billion-parameter model while keeping all base weights fixed. |
Linear Transformers with Learnable Kernel Functions are Better In-Context Models (2024.acl-long)
Copied to clipboard
Yaroslav Aksenov, Nikita Balagansky, Sofia Lo Cicero Vaina, Boris Shaposhnikov, Alexey Gorbatovski, Daniil Gavrilov
| Challenge: | Current Language Models (LMs) lack essential In-Context Learning capabilities, a domain where the Transformer excels. |
| Approach: | They propose a Linear Transformer with a kernel inspired by the Taylor expansion of exponential functions, augmented by convolutional networks. |
| Outcome: | The proposed model amplifies its In-Context Learning abilities on the Pile dataset. |
Train One Sparse Autoencoder Across Multiple Sparsity Budgets to Preserve Interpretability and Accuracy (2025.emnlp-main)
Copied to clipboard
Nikita Balagansky, Yaroslav Aksenov, Daniil Laptev, Vadim Kurochkin, Gleb Gerasimov, Nikita Koriagin, Daniil Gavrilov
| Challenge: | Sparse Autoencoders (SAEs) are powerful tools for interpreting neural networks . conventional SAEs are constrained by the fixed sparsity level chosen during training . |
| Approach: | They propose a training objective that trains a single SAE to optimise reconstructions across multiple sparsity levels simultaneously. |
| Outcome: | The proposed objective achieves Pareto-optimal trade-offs between sparsity and explained variance, outperforming traditional SAEs trained at individual sparsities. |