Papers by Aleksandr Panov
AmbiK: Dataset of Ambiguous Tasks in Kitchen Environment (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are used for behavior planning given natural language instructions from the user. |
| Approach: | They propose to use a textual dataset of ambiguous instructions addressed to a robot in a kitchen environment to compare them. |
| Outcome: | The proposed dataset includes 1000 pairs of ambiguous tasks and their unambiguous counterparts, with environment descriptions, clarifying questions and answers, user intents, and task plans. |
Self-Guided Plan Extraction for Instruction-Following Tasks with Goal-Conditional Reinforcement Learning (2026.findings-acl)
Copied to clipboard
| Challenge: | a framework for instruction-following tasks is proposed for instruction following tasks . previous methods rely on expert trajectories and learn directly from the agent's own interactions with the environment without expert supervision. |
| Approach: | They propose a framework for instruction-following tasks that enables a language model to generate and refine high-level plans through a self-learning mechanism. |
| Outcome: | The proposed framework adheres to instructions more strictly than baseline methods while showing strong generalization to previously unseen instructions. |
Fact-Checking the Output of Large Language Models via Token-Level Uncertainty Quantification (2024.findings-acl)
Copied to clipboard
Ekaterina Fadeeva, Aleksandr Rubashevskii, Artem Shelmanov, Sergey Petrakov, Haonan Li, Hamdy Mubarak, Evgenii Tsymbalov, Gleb Kuzmin, Alexander Panchenko, Timothy Baldwin, Preslav Nakov, Maxim Panov
| Challenge: | Large language models are notorious for producing erroneous claims in their output. |
| Approach: | They propose a fact-checking and hallucination detection pipeline based on token-level uncertainty quantification that removes the impact of uncertainty about what claim to generate on the current step and what surface form to use. |
| Outcome: | The proposed method can fact-check the atomic claims in the output of large language models. |
Faithfulness-Aware Uncertainty Quantification for Fact-Checking the Output of Retrieval-Augmented Generation (2026.findings-acl)
Copied to clipboard
Ekaterina Fadeeva, Aleksandr Rubashevskii, Dzianis Piatrashyn, Roman Vashurin, Shehzaad Dhuliawala, Artem Shelmanov, Timothy Baldwin, Preslav Nakov, Mrinmaya Sachan, Maxim Panov
| Challenge: | Existing approaches to mitigating hallucinations conflate factuality with faithfulness to the retrieved evidence, incorrectly labeling factually correct statements as hallucinos . Existing methods to mitigate hallucinics rely on a lack of training data coverage, input ambiguity, and architectural constraints. |
| Approach: | They propose a method for hallucination detection in Large Language Models enhanced with knowledge retrieval based on faithfulness to the retrieved context. |
| Outcome: | The proposed method outperforms unsupervised UQ baselines, RAG-specific methods, and supervised classifiers across multiple tasks and LLMs. |
CrafText Benchmark: Advancing Instruction Following in Complex Multimodal Open-Ended World (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods to assess instruction following in dynamic and uncertain environments are limited and limited in their ability to adapt to the world's volatility and interdependencies. |
| Approach: | They propose a benchmark for evaluating instruction following in a multimodal environment with diverse instructions and dynamic interactions. |
| Outcome: | The proposed method measures an agent’s ability to generalize to novel instruction formulations and dynamically evolving task configurations, providing a rigorous test of both linguistic understanding and adaptive decision-making. |