Papers by Anton Razzhigaev
Through the Looking Glass: Common Sense Consistency Evaluation of Weird Images (2025.naacl-srw)
Copied to clipboard
Elisei Rykov, Kseniia Petrushina, Kseniia Titova, Anton Razzhigaev, Alexander Panchenko, Vasily Konovalov
| Challenge: | Existing methods to measure image common sense inconsistentness are difficult to implement because of their complexity. |
| Approach: | They propose a visual commonsense model that leverages large vision-language models to extract atomic facts from images and a compact attention-pooling classifier to fine-tune it over encoded atomic fact. |
| Outcome: | The proposed method outperforms existing methods on the WHOOPS! and WEIRD datasets while maintaining a compact attention-pooling classifier over encoded atomic facts. |
MEKER: Memory Efficient Knowledge Embedding Representation for Link Prediction and Question Answering (2022.acl-srw)
Copied to clipboard
| Challenge: | Existing methods to embed learning use a standard Neural Networks (NN) backward mechanism, duplicating its memory consumption. |
| Approach: | They propose a memory-efficient KG embedding model that embeds knowledge graphs as 3rd-order binary tensors. |
| Outcome: | The proposed model yields comparable performance on link prediction and KG-based question answering tasks. |
Your Transformer is Secretly Linear (2024.acl-long)
Copied to clipboard
Anton Razzhigaev, Matvey Mikhalchuk, Elizaveta Goncharova, Nikolai Gerasimenko, Ivan Oseledets, Denis Dimitrov, Andrey Kuznetsov
| Challenge: | a novel linear characteristic exclusive to transformer decoders is revealed: embedding transformations between sequential layers exhibit almost perfect linearity. |
| Approach: | They propose a cosine-similarity-based regularization to reduce layer linearity in transformer decoders. |
| Outcome: | The proposed method improves performance metrics on Tiny Stories and SuperGLUE but also decreases the linearity of the models. |
The Shape of Learning: Anisotropy and Intrinsic Dimensions in Transformer-Based Models (2024.findings-eacl)
Copied to clipboard
Anton Razzhigaev, Matvey Mikhalchuk, Elizaveta Goncharova, Ivan Oseledets, Denis Dimitrov, Andrey Kuznetsov
| Challenge: | Embeddings in transformers encode vast amounts of linguistic nuances and patterns. |
| Approach: | They investigate the anisotropy dynamics and intrinsic dimension of embeddings in transformers . they found that transformer decoders exhibit a bell-shaped anisotropie profile . |
| Outcome: | The investigated embeddings exhibit a bell-shaped curve with the highest anisotropy concentrations in the middle layers . the intrinsic dimension increases in the initial phases of training, indicating an expansion into higher-dimensional space. |
LLM-Microscope: Uncovering the Hidden Role of Punctuation in Context Memory of Transformers (2025.findings-naacl)
Copied to clipboard
Anton Razzhigaev, Matvey Mikhalchuk, Temurbek Rahmatullaev, Elizaveta Goncharova, Polina Druzhinina, Ivan Oseledets, Andrey Kuznetsov
| Challenge: | Large Language Models (LLMs) encode and store contextual information, but internal mechanisms are opaque. |
| Approach: | They propose a toolkit that assesses token-level nonlinearity, evaluates contextual memory, visualizes intermediate layer contributions and measures intrinsic dimensionality of representations. |
| Outcome: | The proposed framework assesses token-level nonlinearity, evaluates contextual memory, visualizes intermediate layer contributions, and measures the intrinsic dimensionality of representations. |
Feature-Level Insights into Artificial Text Detection with Sparse Autoencoders (2025.findings-acl)
Copied to clipboard
Kristian Kuznetsov, Laida Kushnareva, Anton Razzhigaev, Polina Druzhinina, Anastasia Voznyuk, Irina Piontkovskaya, Evgeny Burnaev, Serguei Barannikov
| Challenge: | Existing algorithms for AI text detection lack interpretability, limiting their reliability in highstakes applications. |
| Approach: | They extend existing ATD frameworks by using Sparse Autoencoders to extract features from Gemma-2-2b residual stream. |
| Outcome: | The proposed algorithms can extract human-interpretable features from Gemma-2-2b model. |
Kandinsky: An Improved Text-to-Image Synthesis with Image Prior and Latent Diffusion (2023.emnlp-demo)
Copied to clipboard
Anton Razzhigaev, Arseniy Shakhmatov, Anastasia Maltseva, Vladimir Arkhipkin, Igor Pavlov, Ilya Ryabov, Angelina Kuts, Alexander Panchenko, Andrey Kuznetsov, Denis Dimitrov
| Challenge: | Experimental evaluations demonstrate FID score of 8.03 on the COCO-30K dataset, marking our model as the top open source performer in terms of measurable image generation quality. |
| Approach: | They propose a latent diffusion-based model that combines image prior and latent diffusive techniques to create a text-to-image architecture. |
| Outcome: | The proposed model achieves the highest FID score among open-source models . it is compared with the state-of-the-art models on the COCO-30K dataset . |
A System for Answering Simple Questions in Multiple Languages (2023.acl-demo)
Copied to clipboard
| Challenge: | Existing knowledge graph question answering systems are limited to simple questions, but they can be used to answer complex questions. |
| Approach: | They propose a multilingual Knowledge Graph Question Answering technique that orders potential responses based on the distance between the question’s text embeddings and the answer’s graph embedds. |
| Outcome: | The proposed method consistently outperforms baseline systems, including seq2seq QA models and complex rule-based pipelines. |