Papers by Emelyanov Anton
Humans Keep It One Hundred: an Overview of AI Journey (2020.lrec-1)
Copied to clipboard
Tatiana Shavrina, Anton Emelyanov, Alena Fenogenova, Vadim Fomin, Vladislav Mikhailov, Andrey Evlampiev, Valentin Malykh, Vladimir Larin, Alex Natekin, Aleksandr Vatulin, Peter Romov, Daniil Anastasiev, Nikolai Zinov, Andrey Chertok
| Challenge: | Artificial General Intelligence (AGI) is showing growing performance in numerous applications - beating human performance in Chess and Go, using knowledge bases and text sources to answer questions and even pass human examination. |
| Approach: | They propose to use knowledge bases and text sources to answer questions to improve AI performance on knowledge bases, reasoning and text generation. |
| Outcome: | The proposed AI Journey system passed the final native language exam in Russian with a high score of 69%, with 68% being an average human result. |
FiMMIA: scaling semantic perturbation-based membership inference across modalities (2026.eacl-demo)
Copied to clipboard
| Challenge: | Membership Inference attacks aim to determine whether a specific data point was included in the training set of a target model. |
| Approach: | They propose to train a neural network to analyze the target model’s behavior on perturbed inputs, capturing interactions between semantic domains and loss values on members and non-members in the local neighborhood of each sample. |
| Outcome: | The proposed methods can detect distribution shifts in existing datasets and release a baseline pipeline to detect them. |
MERA: A Comprehensive LLM Evaluation in Russian (2024.acl-long)
Copied to clipboard
Alena Fenogenova, Artem Chervyakov, Nikita Martynov, Anastasia Kozlova, Maria Tikhonova, Albina Akhmetgareeva, Anton Emelyanov, Denis Shevelev, Pavel Lebedev, Leonid Sinev, Ulyana Isaeva, Katerina Kolomeytseva, Daniil Moskovskiy, Elizaveta Goncharova, Nikita Savushkin, Polina Mikhailova, Anastasia Minaeva, Denis Dimitrov, Alexander Panchenko, Sergey Markov
| Challenge: | Recent advances in foundation models have led to the emergence of powerful Large Language Models (LLMs), which showcase unprecedented tasksolving capabilities. |
| Approach: | They propose a method to evaluate FMs and LMs in fixed zero- and few-shot instruction settings that can be extended to other modalities. |
| Outcome: | The proposed evaluation methodology includes an open-source code base and a leaderboard with a submission system. |
Multimodal Evaluation of Russian-language Architectures (2026.eacl-long)
Copied to clipboard
Artem Chervyakov, Ulyana Isaeva, Anton Emelyanov, Artem Safin, Maria Tikhonova, Alexander Kharitonov, Yulia Lyakh, Petr Surovtsev, Denis Shevelev, Vildan Saburov, Vasily Konovalov, Elisei Rykov, Ivan Sviridov, Amina Miftakhova, Ilseyar Alimova, Alexander Panchenko, Alexander Kapitanov, Alena Fenogenova
| Challenge: | Multimodal large language models (MLLMs) are at the center of research attention, yet intelligence, limitations, and risks remain insufficiently understood. |
| Approach: | They propose an open multimodal evaluation framework for Russian-spoken architectures . the framework is instruction-based and includes 18 newly constructed evaluation tasks . |
| Outcome: | The proposed framework provides a replicable methodology for constructing multimodal benchmarks in Russian-spoken architectures. |
RussianSuperGLUE: A Russian Language Understanding Evaluation Benchmark (2020.emnlp-main)
Copied to clipboard
Tatiana Shavrina, Alena Fenogenova, Emelyanov Anton, Denis Shevelev, Ekaterina Artemova, Valentin Malykh, Vladislav Mikhailov, Maria Tikhonova, Andrey Chertok, Andrey Evlampiev
| Challenge: | Modern scientific methodology is beginning to explore universal transformers as an independent object of study. |
| Approach: | They propose a Russian general language understanding evaluation benchmark - Russian SuperGLUE . they provide a benchmark of nine tasks, human level evaluation and a leaderboard for the Russian language . |
| Outcome: | The proposed benchmark provides nine tasks for the Russian language and human level evaluation and leaderboard of transformer models. |