Papers by Artem Vazhentsev
ThinkBooster: A Unified Framework for Seamless Test-Time Scaling of LLM Reasoning (2026.acl-demo)
Copied to clipboard
Vladislav Smirnov, Quang-Chieu Nguyen, Sergey Senichev, Minh Ngoc Ta, Ekaterina Fadeeva, Artem Vazhentsev, Daria Galimzianova, Nikolai Rozanov, Viktor Mazanov, Jingwei Ni, Tianyi Wu, Igor Kiselev, Mrinmaya Sachan, Iryna Gurevych, Preslav Nakov, Timothy Baldwin, Artem Shelmanov
| Challenge: | Existing TTC scaling strategies and reasoning scorers are fragmented and evaluated under inconsistent protocols. |
| Approach: | They propose a framework for seamless test-time compute scaling of large language model reasoning . they use a modular Python library to implement state-of-the-art scaling strategy and scorer families . |
| Outcome: | The proposed framework evaluates performance and computational efficiency on mathematical and coding tasks. |
Token-Level Density-Based Uncertainty Quantification Methods for Eliciting Truthfulness of Large Language Models (2025.naacl-long)
Copied to clipboard
Artem Vazhentsev, Lyudmila Rvanova, Ivan Lazichny, Alexander Panchenko, Maxim Panov, Timothy Baldwin, Artem Shelmanov
| Challenge: | Uncertainty quantification (UQ) is a prominent approach for eliciting truthful answers from large language models (LLMs). |
| Approach: | They propose to use a well-established method for text generation to extract token embeddings from multiple layers of LLMs and compute MD scores for each token. |
| Outcome: | The proposed method improves on existing methods and provides accurate and computationally efficient uncertainty scores for sequence-level selective generation and claim-level fact-checking tasks. |
Hybrid Uncertainty Quantification for Selective Text Classification in Ambiguous Tasks (2023.acl-long)
Copied to clipboard
Artem Vazhentsev, Gleb Kuzmin, Akim Tsvigun, Alexander Panchenko, Maxim Panov, Mikhail Burtsev, Artem Shelmanov
| Challenge: | Existing methods for text classification tasks are inherently ambiguous and can cause errors. |
| Approach: | They propose a method that combines epistemic and aleatoric uncertainty to estimate toxicity detection errors. |
| Outcome: | The proposed method outperforms existing methods for toxicity detection and other ambiguous text classification tasks. |
LM-Polygraph: Uncertainty Estimation for Language Models (2023.emnlp-demo)
Copied to clipboard
Ekaterina Fadeeva, Roman Vashurin, Akim Tsvigun, Artem Vazhentsev, Sergey Petrakov, Kirill Fedyanin, Daniil Vasilev, Elizaveta Goncharova, Alexander Panchenko, Maxim Panov, Timothy Baldwin, Artem Shelmanov
| Challenge: | Large language models often "hallucinate" i.e., fabricate facts without providing users an apparent means to discern the veracity of their statements. |
| Approach: | They propose a framework with implementations of state-of-the-art UE methods for LLMs with unified program interfaces in Python. |
| Outcome: | The proposed framework implements state-of-the-art UE methods for LLMs with unified program interfaces in Python and an extendable benchmark for consistent evaluation by researchers. |
Uncertainty Estimation of Transformer Predictions for Misclassification Detection (2022.acl-long)
Copied to clipboard
Artem Vazhentsev, Gleb Kuzmin, Artem Shelmanov, Akim Tsvigun, Evgenii Tsymbalov, Kirill Fedyanin, Maxim Panov, Alexander Panchenko, Gleb Gusev, Mikhail Burtsev, Manvel Avetisian, Leonid Zhukov
| Challenge: | Uncertainty estimation (UE) of model predictions is crucial step for a variety of tasks such as active learning, misclassification detection, adversarial attack detection, etc. |
| Approach: | They propose to modify UE methods for Transformer models for misclassification detection in named entity recognition and text classification tasks to improve model expressiveness and computational performance. |
| Outcome: | The proposed methods outperform computationally intensive methods on misclassification detection tasks and are based on a large dataset of simulated datasets. |
Uncertainty Quantification for Large Language Models (2025.acl-tutorials)
Copied to clipboard
| Challenge: | Large language models (LLMs) produce hallucinations, which undermine user trust and reliability. |
| Approach: | This tutorial offers the first systematic introduction to uncertainty quantification (UQ) for LLMs in text generation tasks. |
| Outcome: | The proposed framework provides tools for communicating the reliability of a model answer. |
Efficient Out-of-Domain Detection for Sequence to Sequence Models (2023.findings-acl)
Copied to clipboard
Artem Vazhentsev, Akim Tsvigun, Roman Vashurin, Sergey Petrakov, Daniil Vasilev, Maxim Panov, Alexander Panchenko, Artem Shelmanov
| Challenge: | Sequence-to-sequence (seq2sequ) models are a ubiquitous tool for text generation but they are not suitable for many other tasks. |
| Approach: | They propose to use UE techniques to identify out-of-domain (OOD) inputs where the model is susceptible to errors. |
| Outcome: | The proposed methods outperform heavyweight ensembles on the task of OOD detection. |
Unconditional Truthfulness: Learning Unconditional Uncertainty of Large Language Models (2025.emnlp-main)
Copied to clipboard
Artem Vazhentsev, Ekaterina Fadeeva, Rui Xing, Gleb Kuzmin, Ivan Lazichny, Alexander Panchenko, Preslav Nakov, Timothy Baldwin, Maxim Panov, Artem Shelmanov
| Challenge: | Uncertainty quantification (UQ) is a promising approach for detecting hallucinations and low-quality outputs of Large Language Models (LLMs). |
| Approach: | They propose to learn conditional dependency between autoregressive LLM generation steps from attention-based features and a two-staged training procedure to incorporate recurrent features. |
| Outcome: | The proposed method is highly effective for selective generation, achieving substantial improvements over rivaling unsupervised and supervised approaches. |
A Head to Predict and a Head to Question: Pre-trained Uncertainty Quantification Heads for Hallucination Detection in LLM Outputs (2025.emnlp-main)
Copied to clipboard
Artem Shelmanov, Ekaterina Fadeeva, Akim Tsvigun, Ivan Tsvigun, Zhuohan Xie, Igor Kiselev, Nico Daheim, Caiqi Zhang, Artem Vazhentsev, Mrinmaya Sachan, Preslav Nakov, Timothy Baldwin
| Challenge: | Uncertainty quantification (UQ) is a framework for assessing the reliability of model outputs. |
| Approach: | They introduce pre-trained UQ heads for LLMs that are highly robust and generalized to languages they were not explicitly trained on. |
| Outcome: | The pre-trained heads significantly improve their ability to capture uncertainty compared to unsupervised methods. |
When Models Lie, We Learn: Multilingual Span-Level Hallucination Detection with PsiloQA (2025.findings-emnlp)
Copied to clipboard
Elisei Rykov, Kseniia Petrushina, Maksim Savkin, Valerii Olisov, Artem Vazhentsev, Kseniia Titova, Alexander Panchenko, Vasily Konovalov, Julia Belikova
| Challenge: | Existing hallucination detection benchmarks operate at the sequence level and are limited to English . Existing methods lacking fine-grained, multilingual supervision are limited in English based on the sequence . |
| Approach: | They propose a large-scale, multilingual dataset annotated with span-level hallucinations across 14 languages. |
| Outcome: | The proposed dataset annotated with span-level hallucinations across 14 languages is scalable and cost-efficient. |
ALToolbox: A Set of Tools for Active Learning Annotation of Natural Language Texts (2022.emnlp-demos)
Copied to clipboard
Akim Tsvigun, Leonid Sanochkin, Daniil Larionov, Gleb Kuzmin, Artem Vazhentsev, Ivan Lazichny, Nikita Khromov, Danil Kireev, Aleksandr Rubashevskii, Olga Shahmatova, Dmitry V. Dylov, Igor Galitskiy, Artem Shelmanov
| Challenge: | Currently, the framework supports text classification, sequence tagging, and seq2seq tasks. |
| Approach: | They propose an open-source framework for active learning annotation in natural language processing that provides an easy-to-deploy GUI annotation tool directly in the Jupyter IDE. |
| Outcome: | The proposed framework reduces computational overhead and duration of AL iterations and increases annotated data reusability. |