Papers by Vasily Konovalov
Through the Looking Glass: Common Sense Consistency Evaluation of Weird Images (2025.naacl-srw)
Copied to clipboard
Elisei Rykov, Kseniia Petrushina, Kseniia Titova, Anton Razzhigaev, Alexander Panchenko, Vasily Konovalov
| Challenge: | Existing methods to measure image common sense inconsistentness are difficult to implement because of their complexity. |
| Approach: | They propose a visual commonsense model that leverages large vision-language models to extract atomic facts from images and a compact attention-pooling classifier to fine-tune it over encoded atomic fact. |
| Outcome: | The proposed method outperforms existing methods on the WHOOPS! and WEIRD datasets while maintaining a compact attention-pooling classifier over encoded atomic facts. |
Fine-Grained Semantic Comparison of Legal Documents using LLMs (2026.acl-srw)
Copied to clipboard
Elisei Rykov, Nikolay Ivanov, Maria Bandulevich, Kseniia Petrushina, Valentin Malykh, Vasily Konovalov, Alexander Panchenko, Ilseyar Alimova
| Challenge: | Existing tools for detecting inconsistencies and contradictions in complex regulatory documents rely on character-level diffs. |
| Approach: | They propose a benchmark to evaluate span-aware semantic comparison of legal documents . legDiff is an annotated pair of legal paragraphs that is automatically generated . |
| Outcome: | The proposed benchmark evaluates span-aware semantic comparisons of legal documents . it generates synthetic training data that aligns with the manual annotations and mirrors the structure and label distribution of the benchmark . |
DeepPavlov 1.0: Your Gateway to Advanced NLP Models Backed by Transformers and Transfer Learning (2024.emnlp-demo)
Copied to clipboard
Maksim Savkin, Anastasia Voznyuk, Fedor Ignatov, Anna Korzanova, Dmitry Karpov, Alexander Popov, Vasily Konovalov
| Challenge: | Open-source framework for using NLP models is released for non-experts . complexity of building, fine-tuning and deploying state-of-the-art models remains a barrier . |
| Approach: | They present DeepPavlov 1.0, an open-source framework for using NLP models . the framework is based on PyTorch and supports HuggingFace transformers . |
| Outcome: | The DeepPavlov 1.0 framework is designed for practitioners with limited knowledge of NLP/ML. |
Adaptive Retrieval Without Self-Knowledge? Bringing Uncertainty Back Home (2025.acl-long)
Copied to clipboard
Viktor Moskvoretskii, Maria Marina, Mikhail Salnikov, Nikolay Ivanov, Sergey Pletenev, Daria Galimzianova, Nikita Krayko, Vasily Konovalov, Irina Nikishina, Alexander Panchenko
| Challenge: | Recent adaptive retrieval methods integrate LLMs’ intrinsic knowledge with external information appealing to LLM self-knowledge, but they often neglect efficiency evaluations and comparisons with uncertainty estimation techniques. |
| Approach: | They propose to integrate LLMs’ intrinsic knowledge with external information appealing to LLM self-knowledge but neglect efficiency evaluations and comparisons with uncertainty estimation techniques. |
| Outcome: | The proposed methods outperform complex pipelines in terms of efficiency and self-knowledge while maintaining comparable QA performance. |
Will It Still Be True Tomorrow? Multilingual Evergreen Question Classification to Improve Trustworthy QA (2025.emnlp-main)
Copied to clipboard
Sergey Pletenev, Maria Marina, Nikolay Ivanov, Daria Galimzianova, Nikita Krayko, Mikhail Salnikov, Vasily Konovalov, Alexander Panchenko, Viktor Moskvoretskii
| Challenge: | Large Language Models (LLMs) often struggle with question answering due to hallucinated answers. |
| Approach: | They propose a multilingual QA dataset with evergreen labels that can be used to evaluate and train large language models. |
| Outcome: | The proposed model performs well on 12 modern LLMs and EG-E5 classifiers. |
LLM-Independent Adaptive RAG: Let the Question Speak for Itself (2025.emnlp-main)
Copied to clipboard
Maria Marina, Nikolay Ivanov, Sergey Pletenev, Mikhail Salnikov, Daria Galimzianova, Nikita Krayko, Vasily Konovalov, Alexander Panchenko, Viktor Moskvoretskii
| Challenge: | Existing methods to retrieve Large Language Models (LLMs) are inefficient and impractical. |
| Approach: | They propose a lightweight adaptive retrieval method that leverages external information to achieve comparable quality while achieving significant efficiency gains. |
| Outcome: | The proposed methods achieve comparable quality while achieving significant efficiency gains on 6 QA datasets. |
SPY: Enhancing Privacy with Synthetic PII Detection Dataset (2025.naacl-srw)
Copied to clipboard
| Challenge: | Historically, Named Entity Recognition (NER) has been employed for PII detection, but PI I entities constitute a subset of NER entities. |
| Approach: | They propose to use Large Language Models to generate a synthetic dataset that emulates real-world PII scenarios and validate its quality. |
| Outcome: | The proposed dataset is validated and provides a benchmark for PII detection. |
DeepPavlov Strikes Back: A Toolkit for Improving LLM Reliability and Trustworthiness (2026.eacl-demo)
Copied to clipboard
| Challenge: | a new version of an open-source library for natural language processing (NLP) is released . the library is easy to use, flexible, and works with many languages. |
| Approach: | They introduce a new version of an open-source library for natural language processing (NLP) it supports both traditional NLP tasks and new tasks needed to enhance LLMs truthfulness and reliability. |
| Outcome: | The new version of DeepPavlov supports both traditional and new tasks . it is easy to use, flexible, and works with many languages . |
Efficient Answer Retrieval System (EARS): Combining Local DB Search and Web Search for Generative QA (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Developing a virtual assistant is crucial for supporting clients as it provides 24/7 assistance . factual questionanswering system is capable of handling all user queries . |
| Approach: | They propose a production-ready factual question answering system that combines local knowledge base search with generative, context-based QA. |
| Outcome: | The proposed system boosts local knowledge base retrieval by 23% . the system is language-agnostic and can be applied to any data domain . |
How Much Knowledge Can You Pack into a LoRA Adapter without Harming LLM? (2025.findings-naacl)
Copied to clipboard
Sergey Pletenev, Maria Marina, Daniil Moskovskiy, Vasily Konovalov, Pavel Braslavski, Alexander Panchenko, Mikhail Salnikov
| Challenge: | Low-rank adaptation (LoRA) is a popular training technique for updating or domain-specific adaptation of Large Language Models (LLMs). |
| Approach: | They propose to use low-rank adaptation to incorporate new facts into the LLM without compromising previously learned knowledge. |
| Outcome: | The proposed approach is harmful because the model's performance declines after such fine-tuning. |
When Models Lie, We Learn: Multilingual Span-Level Hallucination Detection with PsiloQA (2025.findings-emnlp)
Copied to clipboard
Elisei Rykov, Kseniia Petrushina, Maksim Savkin, Valerii Olisov, Artem Vazhentsev, Kseniia Titova, Alexander Panchenko, Vasily Konovalov, Julia Belikova
| Challenge: | Existing hallucination detection benchmarks operate at the sequence level and are limited to English . Existing methods lacking fine-grained, multilingual supervision are limited in English based on the sequence . |
| Approach: | They propose a large-scale, multilingual dataset annotated with span-level hallucinations across 14 languages. |
| Outcome: | The proposed dataset annotated with span-level hallucinations across 14 languages is scalable and cost-efficient. |
Multimodal Evaluation of Russian-language Architectures (2026.eacl-long)
Copied to clipboard
Artem Chervyakov, Ulyana Isaeva, Anton Emelyanov, Artem Safin, Maria Tikhonova, Alexander Kharitonov, Yulia Lyakh, Petr Surovtsev, Denis Shevelev, Vildan Saburov, Vasily Konovalov, Elisei Rykov, Ivan Sviridov, Amina Miftakhova, Ilseyar Alimova, Alexander Panchenko, Alexander Kapitanov, Alena Fenogenova
| Challenge: | Multimodal large language models (MLLMs) are at the center of research attention, yet intelligence, limitations, and risks remain insufficiently understood. |
| Approach: | They propose an open multimodal evaluation framework for Russian-spoken architectures . the framework is instruction-based and includes 18 newly constructed evaluation tasks . |
| Outcome: | The proposed framework provides a replicable methodology for constructing multimodal benchmarks in Russian-spoken architectures. |