Papers by Boris Shaposhnikov
Steering LLM Reasoning Through Bias-Only Adaptation (2025.emnlp-main)
Copied to clipboard
Viacheslav Sinii, Alexey Gorbatovski, Artem Cherepanov, Boris Shaposhnikov, Nikita Balagansky, Daniil Gavrilov
| Challenge: | Compared with LoRA and BitFit, training a single steering vector per layer with reinforcement learning requires orders of magnitude fewer resources and isolates a much smaller, more interpretable parameter set. |
| Approach: | They propose to train a single steering vector per layer with reinforcement learning while freezing all base weights to match the accuracy of fully RL-tuned reasoning models. |
| Outcome: | The proposed approach improves on an 8 billion-parameter model while keeping all base weights fixed. |
Linear Transformers with Learnable Kernel Functions are Better In-Context Models (2024.acl-long)
Copied to clipboard
Yaroslav Aksenov, Nikita Balagansky, Sofia Lo Cicero Vaina, Boris Shaposhnikov, Alexey Gorbatovski, Daniil Gavrilov
| Challenge: | Current Language Models (LMs) lack essential In-Context Learning capabilities, a domain where the Transformer excels. |
| Approach: | They propose a Linear Transformer with a kernel inspired by the Taylor expansion of exponential functions, augmented by convolutional networks. |
| Outcome: | The proposed model amplifies its In-Context Learning abilities on the Pile dataset. |