Papers by Valentin Malykh
Fine-Grained Semantic Comparison of Legal Documents using LLMs (2026.acl-srw)
Copied to clipboard
Elisei Rykov, Nikolay Ivanov, Maria Bandulevich, Kseniia Petrushina, Valentin Malykh, Vasily Konovalov, Alexander Panchenko, Ilseyar Alimova
| Challenge: | Existing tools for detecting inconsistencies and contradictions in complex regulatory documents rely on character-level diffs. |
| Approach: | They propose a benchmark to evaluate span-aware semantic comparison of legal documents . legDiff is an annotated pair of legal paragraphs that is automatically generated . |
| Outcome: | The proposed benchmark evaluates span-aware semantic comparisons of legal documents . it generates synthetic training data that aligns with the manual annotations and mirrors the structure and label distribution of the benchmark . |
SumTitles: a Summarization Dataset with Low Extractiveness (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods for extractive summarization of dialogue data are limited by the grammar and structure of the utterances used. |
| Approach: | They propose a low-extractive corpus of movie dialogues for abstractive text summarization . they use an alignment algorithm to construct the corpus and a baseline evaluation . |
| Outcome: | The proposed method is low-extractive and shows high performance in dialogue datasets. |
StRuCom: A Novel Dataset of Structured Code Comments in Russian (2025.acl-srw)
Copied to clipboard
| Challenge: | Existing machine learning models for code comment generation are poorly suited for Russian . existing datasets that contain simple comments and docstrings in English are not suitable for function-level documentation generation. |
| Approach: | They propose a dataset specifically designed for Russian code documentation. |
| Outcome: | The first large-scale dataset specifically designed for Russian code documentation is based on human-written comments from GitHub repositories with synthetically generated ones. |
Humans Keep It One Hundred: an Overview of AI Journey (2020.lrec-1)
Copied to clipboard
Tatiana Shavrina, Anton Emelyanov, Alena Fenogenova, Vadim Fomin, Vladislav Mikhailov, Andrey Evlampiev, Valentin Malykh, Vladimir Larin, Alex Natekin, Aleksandr Vatulin, Peter Romov, Daniil Anastasiev, Nikolai Zinov, Andrey Chertok
| Challenge: | Artificial General Intelligence (AGI) is showing growing performance in numerous applications - beating human performance in Chess and Go, using knowledge bases and text sources to answer questions and even pass human examination. |
| Approach: | They propose to use knowledge bases and text sources to answer questions to improve AI performance on knowledge bases, reasoning and text generation. |
| Outcome: | The proposed AI Journey system passed the final native language exam in Russian with a high score of 69%, with 68% being an average human result. |
DeepPavlov: Open-Source Library for Dialogue Systems (P18-4)
Copied to clipboard
Mikhail Burtsev, Alexander Seliverstov, Rafael Airapetyan, Mikhail Arkhipov, Dilyara Baymurzina, Nickolay Bushkov, Olga Gureenkova, Taras Khakhulin, Yuri Kuratov, Denis Kuznetsov, Alexey Litinsky, Varvara Logacheva, Alexey Lymar, Valentin Malykh, Maxim Petrov, Vadim Polulyakh, Leonid Pugachev, Alexey Sorokin, Maria Vikhreva, Marat Zaynutdinov
| Challenge: | open-source library DeepPavlov is designed for rapid development of dialogue systems. |
| Approach: | open-source library DeepPavlov is tailored for development of conversational agents . the library prioritizes efficiency, modularity and extensibility with the goal to make it easier to develop dialogue systems from scratch . |
| Outcome: | the open-source library DeepPavlov is designed for rapid development of dialogue systems . it supports modular as well as end-to-end approaches to implementation of conversational agents . |
DRAGOn: Designing RAG On Periodically Updated Corpus (2026.eacl-srw)
Copied to clipboard
Fedor Chernogorskii, Sergei Averkiev, Liliya Kudraleeva, Zaven Martirosian, Maria Tikhonova, Valentin Malykh, Alena Fenogenova
| Challenge: | Existing methods for evaluating RAG systems are labor-intensive and difficult to maintain. |
| Approach: | They propose a method to design a RAG benchmark on a regularly updated corpus. |
| Outcome: | The proposed method uses a regularly updated corpus to evaluate RAG models. |
CCT-Code: Cross-Consistency Training for Multilingual Clone Detection and Code Search (2025.naacl-srw)
Copied to clipboard
Nikita Sorokin, Tikhonov Anton, Dmitry Abulkhanov, Ivan Sedykh, Irina Piontkovskaya, Valentin Malykh
| Challenge: | clone detection is crucial in software development for identifying semantically similar code . clones can be found in the same language code snippets, but there is little research on multilingual clonage detection. |
| Approach: | They propose a novel training procedure leveraging cross-lingual similarity to train language models on source code in various programming languages. |
| Outcome: | The proposed method achieves state-of-the-art on C++ and Python clone detection benchmarks with comparable performance on decoder-based models. |
Searching by Code: A New SearchBySnippet Dataset and SnippeR Retrieval Model for Searching by Code Snippets (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing code search algorithms use code comments rather than full-text descriptions as text . existing algorithms use a code snippet and/or error traceback to find code . |
| Approach: | They propose a new search-by-code use case using a code snippet and error traceback . they propose implementing the search- by-code query in a StackOverflow dataset . |
| Outcome: | The proposed dataset outperforms strong baselines on SearchBySnippet with 0.451 Recall@10 . a code snippet and/or error traceback are used as queries to find bugs . |
Low-resource Machine Translation for Code-switched Kazakh-Russian Language Pair (2025.naacl-srw)
Copied to clipboard
| Challenge: | Existing methods to train machine translation models for low-resource languages are not available. |
| Approach: | They propose to build a machine translation model for code-switched Kazakh-Russian language pair with no labeled data. |
| Outcome: | The proposed method beats an existing commercial system by human evaluation on a Kazakh-Russian language pair with no labeled data. |
Robust to Noise Models in Natural Language Processing Tasks (P19-2)
Copied to clipboard
| Challenge: | Existing spelling correction systems are far from perfect for noise-sensitive texts . a new way to handle noise is to make models robust to noise. |
| Approach: | They propose a robust to noise word embeddings model which outperforms existing models in different tasks. |
| Outcome: | The proposed model outperforms existing models in three downstream tasks and shows improvements in noise robustness over existing models. |
SWE-MERA: A Dynamic Benchmark for Agenticly Evaluating Large Language Models on Software Engineering Tasks (2025.emnlp-demos)
Copied to clipboard
Adamenko Pavel, Ivanov Mikhail, Aidar Valeev, Rodion Levichev, Pavel Zadorozhny, Ivan Lopatin, Dmitrii Babaev, Alena Fenogenova, Valentin Malykh
| Challenge: | SWE-bench is a static benchmark that collects only once and never updates. |
| Approach: | They propose a dynamic, continuously updated benchmark to address data contamination issues by collecting real-world GitHub issues and rigorous quality validation. |
| Outcome: | The proposed benchmarks are based on a dataset of 2,294 GitHub issues and their corresponding pull requests (PRs) the static nature of the benchmarks makes it hard to distinguish meaningful progress. |
Call, Reward, Repeat: Advancing Dialog State Tracking with GRPO and Function Calling (2026.eacl-srw)
Copied to clipboard
| Challenge: | Recent advances in Large Language Models (LLMs) have notably enhanced task-oriented dialogue systems, particularly in Dialogue State Tracking (DST). |
| Approach: | They propose a group-relative policy optimization method that guides LLMs toward improved DST accuracy even under low-resource conditions. |
| Outcome: | The proposed method improves on established DST benchmarks while using significantly reduced out-of-domain training data. |
A System for Answering Simple Questions in Multiple Languages (2023.acl-demo)
Copied to clipboard
| Challenge: | Existing knowledge graph question answering systems are limited to simple questions, but they can be used to answer complex questions. |
| Approach: | They propose a multilingual Knowledge Graph Question Answering technique that orders potential responses based on the distance between the question’s text embeddings and the answer’s graph embedds. |
| Outcome: | The proposed method consistently outperforms baseline systems, including seq2seq QA models and complex rule-based pipelines. |
Ask Me Anything in Your Native Language (2022.naacl-main)
Copied to clipboard
| Challenge: | Cross-lingual question answering systems are becoming more and more important . a new approach can be generalized to more than 20 languages and outperforms previous models by 12% . |
| Approach: | They propose a cross-lingual question answering system that can be generalized to more than 20 languages . their approach can outperform previous models by 12% on multiple languages based on a dataset . |
| Outcome: | The proposed approach outperforms the previous models on multiple languages by 12% . it can be generalized to more than 20 languages and outperformed all previous models by 2% . |
RussianSuperGLUE: A Russian Language Understanding Evaluation Benchmark (2020.emnlp-main)
Copied to clipboard
Tatiana Shavrina, Alena Fenogenova, Emelyanov Anton, Denis Shevelev, Ekaterina Artemova, Valentin Malykh, Vladislav Mikhailov, Maria Tikhonova, Andrey Chertok, Andrey Evlampiev
| Challenge: | Modern scientific methodology is beginning to explore universal transformers as an independent object of study. |
| Approach: | They propose a Russian general language understanding evaluation benchmark - Russian SuperGLUE . they provide a benchmark of nine tasks, human level evaluation and a leaderboard for the Russian language . |
| Outcome: | The proposed benchmark provides nine tasks for the Russian language and human level evaluation and leaderboard of transformer models. |