Papers by Maria Tikhonova
A Family of Pretrained Transformer Language Models for Russian (2024.lrec-main)
Copied to clipboard
Dmitry Zmitrovich, Aleksandr Abramov, Andrey Kalmykov, Vitaly Kadulin, Maria Tikhonova, Ekaterina Taktasheva, Danil Astafurov, Mark Baushenko, Artem Snegirev, Tatiana Shavrina, Sergei S. Markov, Vladislav Mikhailov, Alena Fenogenova
| Challenge: | Developing Transformer language models for the Russian language has received little attention . most of these LMs are developed for English, which imposes substantial constraints on the potential of the language technologies. |
| Approach: | They propose to release 13 Russian Transformer language models that span three languages . they aim to broaden the scope of NLP research directions and develop industrial solutions for the Russian language. |
| Outcome: | The proposed models are based on Russian language datasets and benchmarks. |
DRAGOn: Designing RAG On Periodically Updated Corpus (2026.eacl-srw)
Copied to clipboard
Fedor Chernogorskii, Sergei Averkiev, Liliya Kudraleeva, Zaven Martirosian, Maria Tikhonova, Valentin Malykh, Alena Fenogenova
| Challenge: | Existing methods for evaluating RAG systems are labor-intensive and difficult to maintain. |
| Approach: | They propose a method to design a RAG benchmark on a regularly updated corpus. |
| Outcome: | The proposed method uses a regularly updated corpus to evaluate RAG models. |
TAPE: Assessing Few-shot Russian Language Understanding (2022.findings-emnlp)
Copied to clipboard
Ekaterina Taktasheva, Tatiana Shavrina, Alena Fenogenova, Denis Shevelev, Nadezhda Katricheva, Maria Tikhonova, Albina Akhmetgareeva, Oleg Zinkevich, Anastasiia Bashmakova, Svetlana Iordanskaia, Alena Spiridonova, Valentina Kurenshchikova, Ekaterina Artemova, Vladislav Mikhailov
| Challenge: | Recent advances in zero-shot and few-shot learning have shown promise for a scope of research and practical purposes, but lacks standardized evaluation suites for non-English languages. |
| Approach: | They propose a novel benchmark that includes six more complex NLU tasks for Russian, covering multi-hop reasoning, ethical concepts, logic and commonsense knowledge. |
| Outcome: | The proposed benchmark includes six more complex NLU tasks for Russian, covering multi-hop reasoning, ethical concepts, logic and commonsense knowledge. |
MERA: A Comprehensive LLM Evaluation in Russian (2024.acl-long)
Copied to clipboard
Alena Fenogenova, Artem Chervyakov, Nikita Martynov, Anastasia Kozlova, Maria Tikhonova, Albina Akhmetgareeva, Anton Emelyanov, Denis Shevelev, Pavel Lebedev, Leonid Sinev, Ulyana Isaeva, Katerina Kolomeytseva, Daniil Moskovskiy, Elizaveta Goncharova, Nikita Savushkin, Polina Mikhailova, Anastasia Minaeva, Denis Dimitrov, Alexander Panchenko, Sergey Markov
| Challenge: | Recent advances in foundation models have led to the emergence of powerful Large Language Models (LLMs), which showcase unprecedented tasksolving capabilities. |
| Approach: | They propose a method to evaluate FMs and LMs in fixed zero- and few-shot instruction settings that can be extended to other modalities. |
| Outcome: | The proposed evaluation methodology includes an open-source code base and a leaderboard with a submission system. |
The Russian-focused embedders’ exploration: ruMTEB benchmark and Russian embedding model design (2025.naacl-long)
Copied to clipboard
| Challenge: | Embedding models are used in tasks such as information retrieval and semantic textual similarity. |
| Approach: | They propose a new Russian-focused embedding model called ru-en-RoSBERTa and a benchmark for Russian language . they propose to use the roMTEB benchmark to assess Russian and multilingual models . |
| Outcome: | The proposed model achieves results that are on par with state-of-the-art models in Russian. |
Multimodal Evaluation of Russian-language Architectures (2026.eacl-long)
Copied to clipboard
Artem Chervyakov, Ulyana Isaeva, Anton Emelyanov, Artem Safin, Maria Tikhonova, Alexander Kharitonov, Yulia Lyakh, Petr Surovtsev, Denis Shevelev, Vildan Saburov, Vasily Konovalov, Elisei Rykov, Ivan Sviridov, Amina Miftakhova, Ilseyar Alimova, Alexander Panchenko, Alexander Kapitanov, Alena Fenogenova
| Challenge: | Multimodal large language models (MLLMs) are at the center of research attention, yet intelligence, limitations, and risks remain insufficiently understood. |
| Approach: | They propose an open multimodal evaluation framework for Russian-spoken architectures . the framework is instruction-based and includes 18 newly constructed evaluation tasks . |
| Outcome: | The proposed framework provides a replicable methodology for constructing multimodal benchmarks in Russian-spoken architectures. |
RussianSuperGLUE: A Russian Language Understanding Evaluation Benchmark (2020.emnlp-main)
Copied to clipboard
Tatiana Shavrina, Alena Fenogenova, Emelyanov Anton, Denis Shevelev, Ekaterina Artemova, Valentin Malykh, Vladislav Mikhailov, Maria Tikhonova, Andrey Chertok, Andrey Evlampiev
| Challenge: | Modern scientific methodology is beginning to explore universal transformers as an independent object of study. |
| Approach: | They propose a Russian general language understanding evaluation benchmark - Russian SuperGLUE . they provide a benchmark of nine tasks, human level evaluation and a leaderboard for the Russian language . |
| Outcome: | The proposed benchmark provides nine tasks for the Russian language and human level evaluation and leaderboard of transformer models. |