Papers by Elena Tutubalina
Anatomy of Unlearning: The Dual Impact of Fact Salience and Model Fine-Tuning (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing studies assume that all facts are equally forgettable . popular facts, frequent and widely distributed, may be more deeply embedded than rare ones, making them harder to erase. |
| Approach: | They propose a benchmark to evaluate how unlearning differs between pretrained and supervised fine-tuned models when fact popularity is taken into account. |
| Outcome: | The proposed model is compared with pretrained and SFT models on the forget data and shows that it performs better on both models. |
Fair Evaluation in Concept Normalization: a Large-scale Comparative Analysis for BERT-based Models (2020.coling-main)
Copied to clipboard
| Challenge: | a large number of biomedical entity mentions are retrieved from different ontologies, requiring non-syntactic interpretation. |
| Approach: | They propose to use bidirectional encoder representations from transformers to link biomedical entities across three domains for a task called medical concept normalization. |
| Outcome: | The proposed neural architectures are efficient for linking biomedical entities across domains and corpora. |
POLAR: A Benchmark for Multilingual, Multicultural, and Multi-Event Online Polarization (2026.findings-acl)
Copied to clipboard
Usman Naseem, Robert Geislinger, Juan Ren, Sarah Kohail, Rudy Alexandro Garrido Veliz, P Sam Sahil, Yiran Zhang, Idris Abdulmumin, Marco Antonio Stranisci, Özge Alacam, Cengiz Acarturk, Aisha Jabr, Saba Anwar, Abinew Ali Ayele, Simona Frenda, Alessandra Teresa Cignarella, Elena Tutubalina, Oleg Rogov, Aung Kyaw Htet, Xintong Wang, Surendrabikram Thapa, Kritesh Rauniyar, Tanmoy Chakraborty, MD Arfeen Zeeshan, Dheeraj Kodati, Satya Keerthi, Sahar Moradizeyveh, Firoj Alam, Md Arid Hasan, Syed Ishtiaque Ahmed, Ye Kyaw Thu, Shantipriya Parida, Ihsan Ayyub Qazi, Lilian Diana Awuor Wanzare, Nelson Odhiambo Onyango, Clemencia Siro, Jane Wanjiru Kimani, Ibrahim Said Ahmad, Adem Chanie Ali, Martin Semmann, Chris Biemann, Shamsuddeen Hassan Muhammad, Seid Muhie Yimam
| Challenge: | polarization is a pervasive threat to democratic institutions, civil discourse, and social cohesion worldwide . most existing datasets focus on English or high-resource languages, reflecting a widespread trend across NLP tasks . |
| Approach: | They propose a multilingual, multicultural, and multi-event dataset with over 110K instances in 22 languages drawn from diverse online platforms and real-world events. |
| Outcome: | The proposed dataset analyzes polarization detection, type, and manifestation using a variety of annotation platforms adapted to each cultural context. |
Biomedical Entity Representation with Graph-Augmented Multi-Objective Transformer (2024.findings-naacl)
Copied to clipboard
| Challenge: | Modern biomedical concept representations are mostly trained on synonymous concept names from a biomedically knowledge base graph, ignoring the inter-concept interactions and a concept’s local neighborhood. |
| Approach: | They propose a Graph-Augmented Multi-Objective Transformer which captures both inter-concept and intra-conception interactions from the multilingual UMLS graph. |
| Outcome: | The proposed model captures inter- and intra-concept interactions from the multilingual UMLS graph using pre-trained language models and graph neural networks. |
Detecting Adverse Drug Reactions from Biomedical Texts with Neural Networks (P19-2)
Copied to clipboard
| Challenge: | Detection of adverse drug reactions in post-marketing period is a crucial challenge for pharmacology. |
| Approach: | They propose to use social media to extract information about adverse drug reactions . they compare four state-of-the-art attention-based neural networks to the F-measure . |
| Outcome: | The proposed methods perform better on four different benchmarks. |
One Task Vector is not Enough: A Large-Scale Study for In-Context Learning (2026.acl-srw)
Copied to clipboard
| Challenge: | Existing studies limit comprehensive analysis of large language models based on task vectors . recent work points to "task vectors" as mechanism for encoding task rules . |
| Approach: | They propose a novel task vector with 30 input-output pairs for in-context learning . they use a few prompt-based examples to adapt to new tasks without weight updates . |
| Outcome: | Experiments with Llama-3-8B on QAF show task vector performance peaks at intermediate layer . complex tasks rely on multiple, subtask-specific vectors rather than a single vector . |
Confidence Leaps in LLM Reasoning: Early Stopping and Cross-Model Transfer (2026.eacl-short)
Copied to clipboard
| Challenge: | Large Language Models build confidence gradually during reasoning, but internal dynamics of how confidence evolves during this reasoning process remain poorly understood. |
| Approach: | They propose a model-agnostic early-stopping heuristic that halts generation upon detecting a "confidence leap" they argue that conviction is often reached in a discrete "moment of insight" they propose to train models without sacrificing accuracy . |
| Outcome: | The proposed model-agnostic heuristic reduces generation time without sacrificing accuracy and significantly reduces the generation time. |
Biomedical Concept Normalization over Nested Entities with Partial UMLS Terminology in Russian (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing annotations in Russian do not include all entities, but only a small fraction of them are labeled in English. |
| Approach: | They present a manually annotated PubMed abstract dataset for concept normalization in Russian. |
| Outcome: | The proposed model improves on nested named entities in a zero-shot setting on bilingual terminology. |
SkipCLM: Enhancing Crosslingual Alignment of Decoder Transformer Models via Contrastive Learning and Skip Connection (2025.naacl-srw)
Copied to clipboard
| Challenge: | Existing contrastive learning methods for cross-lingual alignment are not effective for multilingual machine translation tasks. |
| Approach: | They propose a method that augments contrastive learning for cross-lingual alignment with a trainable skip connection to preserve information crucial for accurate target language generation. |
| Outcome: | Experiments with XGLM-564M on the Flores-101 benchmark show that the proposed method preserves crucial information crucial for accurate target language generation. |
CLEAR: Character Unlearning in Textual and Visual Modalities (2025.findings-acl)
Copied to clipboard
Alexey Dontsov, Dmitrii Korzh, Alexey Zhavoronkin, Boris Mikheev, Denis Bobkov, Aibek Alanov, Oleg Rogov, Ivan Oseledets, Elena Tutubalina
| Challenge: | Existing methods for removing private or hazardous data from deep learning models are focused on single-modality models. |
| Approach: | They propose CLEAR, the first open-source benchmark specifically for MMU. CLEAR contains 200 fictitious individuals and 3,700 images linked with corresponding question-answer pairs. |
| Outcome: | The proposed benchmarks show that unlearning both modalities outperform single-modality approaches. |
Two Steps from Hell: Compositionality on Chemical LMs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Experiments with state-of-the-art ChemLLMs show significant performance drops in compositional tasks, highlighting the need for models that move beyond pattern recognition. |
| Approach: | They introduce a benchmark to evaluate chemical language models' understanding of chemical language by identifying and analyzing compositional patterns within chemical data. |
| Outcome: | The proposed benchmark shows that existing LLMs can handle complex queries without pattern recognition. |
Out of Distribution, Out of Luck: Process Rewards Misguide Reasoning Models (2026.eacl-short)
Copied to clipboard
| Challenge: | 80% of reasoning model outputs respond to formatting artifacts rather than mathematical content. |
| Approach: | They evaluate process reward models that provide step-level feedback during inference . they identify distinct reward prediction patterns that differentiate reasoning from non-reasoning model outputs . |
| Outcome: | The proposed model fails to enhance and sometimes degrade reasoning model performance. |
Medical Crossing: a Cross-lingual Evaluation of Clinical Entity Linking (2022.lrec-1)
Copied to clipboard
Anton Alekseev, Zulfat Miftahutdinov, Elena Tutubalina, Artem Shelmanov, Vladimir Ivanov, Vladimir Kokh, Alexander Nesterov, Manvel Avetisian, Andrei Chertok, Sergey Nikolenko
| Challenge: | Existing approaches to medical entity linking are limited in terms of data volume and languages. |
| Approach: | They propose to use clinical reports, clinical guidelines, and medical research papers to evaluate cross-lingual medical entity linking. |
| Outcome: | The proposed model outperforms existing models on clinical reports, clinical guidelines, and medical research papers. |
The Silence of the Facts: Popularity as a Barrier to Machine Unlearning (2026.acl-srw)
Copied to clipboard
| Challenge: | Existing unlearning methods assume that all facts are equally challenging to forget . large models struggle more to forget popular entities, damaging related knowledge in the process . |
| Approach: | They build a benchmark to investigate whether fact popularity influences the efficiency of LLM unlearning. |
| Outcome: | The proposed benchmark compares state-of-the-art models on a set of models of different sizes. |
Entity Linking over Nested Named Entities for Russian (2022.lrec-1)
Copied to clipboard
Natalia Loukachevitch, Pavel Braslavski, Vladimir Ivanov, Tatiana Batura, Suresh Manandhar, Artem Shelmanov, Elena Tutubalina
| Challenge: | Entity linking is a popular NLP task, where a system needs to link a named entity to a concept in a knowledge base such as Wikidata. |
| Approach: | They describe the main design principles behind entity linking annotation in the recently released Russian NEREL dataset for information extraction. |
| Outcome: | The NEREL dataset is the largest Russian dataset annotated with entities and relations. |
RuCCoD: Towards Automated ICD Coding in Russian (2025.emnlp-main)
Copied to clipboard
Alexandr Nesterov, Andrey Sakhovskiy, Ivan Sviridov, Airat Valiev, Vladimir Makharev, Petr Anokhin, Galina Zubkova, Elena Tutubalina
| Challenge: | a new dataset for clinical coding in Russian is available for download . human coders must navigate a wide array of medical terminology and time pressures . |
| Approach: | They present a new dataset for ICD coding in Russian, a language with limited biomedical resources. |
| Outcome: | The proposed model improves accuracy on an in-house EHR dataset from 2017 to 2021. |
Lost in Translation: Chemical Language Models and the Misunderstanding of Molecule Structures (2024.findings-emnlp)
Copied to clipboard
Veronika Ganeeva, Andrey Sakhovskiy, Kuzma Khrabrov, Andrey Savchenko, Artur Kadurin, Elena Tutubalina
| Challenge: | chemistry and natural language processing (NLP) have advanced drug discovery. |
| Approach: | They propose a framework for assessment of Chemistry LMs of different natures that relies on augmentations that preserve an underlying chemical. |
| Outcome: | The proposed framework relies on augmentations that preserve an underlying chemical, such as kekulization and cycle replacements. |
SynthDetoxM: Modern LLMs are Few-Shot Parallel Detoxification Data Annotators (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing approaches to multilingual text detoxification are hampered by the scarcity of parallel multilingual datasets. |
| Approach: | They propose a pipeline for the generation of multilingual parallel detoxification data and a dataset for SynthDetoxM which is manually generated and rewritten with open-source LLMs. |
| Outcome: | The proposed pipeline outperforms human-annotated datasets even in data limited setting. |
PAUQ: Text-to-SQL in Russian (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Semantic parsing is an important task that allows to democratize human-computer interaction. |
| Approach: | They construct and complement a Russian text-to-SQL dataset by integrating a spider query with a RAT-SqL and BRIDGE database. |
| Outcome: | The proposed datasets show that they perform well with monolingual training and improved accuracy in multilingual scenarios. |
Evolutionary Search for Automated Design of Uncertainty Quantification Methods (2026.acl-srw)
Copied to clipboard
Mikhail Seleznyov, Daniil Korbut, Viktor Moskvoretskii, Oleg Somov, Alexander Panchenko, Elena Tutubalina
| Challenge: | Qualitative analysis reveals that different LLMs employ qualitatively distinct evolutionary strategies for automating, interpretable hallucination detector design. |
| Approach: | They apply LLM-powered evolutionary search to discover unsupervised UQ methods represented as Python programs and apply them to atomic claim verification. |
| Outcome: | The proposed methods outperform strong manually-designed baselines while generalizing robustly out-of-distribution. |
SPARTA: Evaluating Reasoning Segmentation Robustness through Black-Box Adversarial Paraphrasing in Text Autoencoder Latent Space (2026.eacl-long)
Copied to clipboard
Viktoriia Zinkovich, Anton Antonov, Andrei Spiridonov, Denis Shepelev, Andrey Moskalenko, Daria Pugacheva, Elena Tutubalina, Andrey Kuznetsov, Vlad Shakhuro
| Challenge: | Existing work on semantically equivalent textual paraphrases has focused on perturbing image inputs. |
| Approach: | They propose a novel adversarial paraphrasing task that generates grammatically correct paraphrases that sighed the original query meaning while degrading segmentation performance. |
| Outcome: | The proposed task outperforms previous methods by up to 2x on ReasonSeg and LLMSeg-40k datasets. |
Ad Lingua: Text Classification Improves Symbolism Prediction in Image Advertisements (2020.coling-main)
Copied to clipboard
Andrey Savchenko, Anton Alekseev, Sejeong Kwon, Elena Tutubalina, Evgeny Myasnikov, Sergey Nikolenko
| Challenge: | a recent study shows that image-based symbols are insufficient for symbolism prediction in visual advertising . a new method is proposed to help understand image advertisements . |
| Approach: | They propose a multimodal image-based classifier and object detection classifier for symbols . they propose 'symbolic' annotation tasks to help users understand ads' |
| Outcome: | The proposed system establishes state-of-the-art in symbolism prediction. |
A Comprehensive Evaluation of Biomedical Entity-centric Search (2022.emnlp-industry)
Copied to clipboard
| Challenge: | a novel algorithm for information retrieval from biomedical abstracts is used to identify entities. |
| Approach: | They perform a fine-grained evaluation of a BERT-based biomedical search engine . they use manually annotated PubMed abstracts and off-she-shelf Elasticsearch . |
| Outcome: | The proposed system performs better for disease and gene search queries than other systems. |
Deep Neural Models for Medical Concept Normalization in User-Generated Texts (P19-2)
Copied to clipboard
| Challenge: | a medical concept normalization problem is a challenge since social media texts are ambiguous and noisy . a recent study shows that neural architectures leverage the semantic meaning of the entity mention . |
| Approach: | They propose to map a health-related entity mention to a controlled vocabulary . they use powerful neural networks and contextualized word representation models . |
| Outcome: | The proposed model outperforms existing state-of-the-art models in mapping medical concepts to medical terms . the proposed model is based on recurrent neural networks and contextualized word representation models . |
Emergent Misalignment via In-Context Learning: Narrow in-context examples can produce broadly misaligned LLMs (2026.acl-long)
Copied to clipboard
Nikita Afonin, Nikita Andriianov, Vahagn Hovhannisyan, Nikhil Bageshpura, Kyle Liu, Kevin Zhu, Sunishchal Dev, Ashwinee Panda, Oleg Rogov, Elena Tutubalina, Alexander Panchenko, Mikhail Seleznyov
| Challenge: | Recent studies have documented emergent misalignment in language models adapted on narrow examples . Emergent misalignement occurs when models are trained on narrow set of misallocated examples resulting in harmful or misleading responses . |
| Approach: | They propose to explain in-context EM as conflict between safety objectives and context-following behavior. |
| Outcome: | The proposed model is adapted on 16 in-context examples and produces misaligned responses to benign queries. |
Feature Drift: How Fine-Tuning Repurposes Representations in LLMs (2026.findings-eacl)
Copied to clipboard
| Challenge: | Sparse autoencoders (SAEs) are a powerful tool for interpreting neural networks by extracting concepts (features) represented in their activations. |
| Approach: | They propose to use Sparse Autoencoders to extract concepts from their activations to explain how fine-tuning changes model capabilities. |
| Outcome: | The proposed model recombines existing concepts rather than learning new ones, and shows that it is a better explanation for how fine-tuning changes model capabilities. |
When Punctuation Matters: A Large-Scale Comparison of Prompt Robustness Methods for LLMs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are sensitive to subtle, non-semantic variations in prompt phrasing and formatting. |
| Approach: | They propose to evaluate 4 methods for improving prompt robustness within a unified experimental framework. |
| Outcome: | The proposed methods are compared to 8 models from Llama, Qwen and Gemma families and are generalized against multiple types of distribution shifts. |
Vote’n’Rank: Revision of Benchmarking with Social Choice Theory (2023.eacl-main)
Copied to clipboard
Mark Rofin, Vladislav Mikhailov, Mikhail Florinsky, Andrey Kravchenko, Tatiana Shavrina, Elena Tutubalina, Daniel Karabekyan, Ekaterina Artemova
| Challenge: | ML benchmarks have been criticized for their construct validity, fragility of the design and task choices. |
| Approach: | They propose a framework for ranking systems in multi-task benchmarks under the principles of the social choice theory and propose 'vote'n'rank' procedures are more robust than the mean average while being able to handle missing performance scores and determine conditions under which the system becomes the winner. |
| Outcome: | The proposed framework can be utilised to draw new insights on benchmarking in several ML sub-fields and identify the best-performing systems in research and development case studies. |
Bring the Apple, Not the Sofa: Impact of Irrelevant Context in Embodied AI Commands on VLA Models (2026.eacl-srw)
Copied to clipboard
Andrey Moskalenko, Daria Pugacheva, Denis Shepelev, Andrey Kuznetsov, Vlad Shakhuro, Elena Tutubalina
| Challenge: | Embodied AI is undergoing rapid development, with robots increasingly exhibiting practical utility in everyday environments. |
| Approach: | They evaluate the robustness of vision language action models under linguistic perturbations . they categorize irrelevant contexts into two groups according to their length and proximity to robot commands . |
| Outcome: | The proposed model can exhibit relative robustness to random context, with a performance drop within 10%, the authors show . human paraphrases of instructions lead to a drop of nearly 20%, the study shows . |
RuCCoN: Clinical Concept Normalization in Russian (2022.findings-acl)
Copied to clipboard
Alexandr Nesterov, Galina Zubkova, Zulfat Miftahutdinov, Vladimir Kokh, Elena Tutubalina, Artem Shelmanov, Anton Alekseev, Manvel Avetisian, Andrey Chertok, Sergey Nikolenko
| Challenge: | a new dataset for clinical concept normalization in Russian is available for download . ehrs contains over 16,028 entity mentions manually linked to over 2,409 unique concepts . |
| Approach: | They present a dataset for clinical concept normalization in Russian manually annotated by medical professionals. |
| Outcome: | The proposed dataset contains 16,028 entity mentions manually linked to over 2,409 unique concepts from the Russian language part of the UMLS ontology. |