Papers by Sherzod Hakimov
Retrieval-Augmented Code Generation for Situated Action Generation: A Case Study on Minecraft (2024.findings-emnlp)
Copied to clipboard
| Challenge: | In the Minecraft Collaborative Building Task, two players collaborate to build a building using 3D blocks. |
| Approach: | They propose to use large language models to model the Builder's sequence of actions in the Minecraft Collaborative Building Task. |
| Outcome: | The proposed methods significantly improve performance over baseline methods and provide detailed analysis for future work. |
M2SA: Multimodal and Multilingual Model for Sentiment Analysis of Tweets (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing studies on sentiment analysis of tweets focus on the English language . however, there is still a challenge of processing lower-resourced languages . |
| Approach: | They transform tweet sentiment dataset into a multimodal format through a straightforward curation process. |
| Outcome: | The proposed approach performs exceptionally well in unimodal and multimodal configurations. |
The Price of Thought: A Multilingual Analysis of Reasoning, Performance, and Cost of Negotiation in Large Language Models (2026.findings-eacl)
Copied to clipboard
Sherzod Hakimov, Roland Bernard, Tim Leiber, Karl Osswald, Kristina Richert, Ruilin Yang, Raffaella Bernardi, David Schlangen
| Challenge: | Negotiation is a fundamental challenge for AI agents as it requires an ability to reason strategically, model opponents, and balance cooperation with competition. |
| Approach: | They propose to use a self-play setup to compare commercial and open-weight large language models to their vanilla counterparts in three different languages to examine trade-offs between performance and cost. |
| Outcome: | The proposed model improves GPT-5's performance by 31.4 % while increasing its cost by nearly 400 %. |
clembench: Using Game Play to Evaluate Chat-Optimized Language Models as Conversational Agents (2023.emnlp-main)
Copied to clipboard
Kranti Chalamalasetti, Jana Götze, Sherzod Hakimov, Brielen Madureira, Philipp Sadler, David Schlangen
| Challenge: | Recent work suggests large language models can be understood as (simulators of) such agents. |
| Approach: | They propose a method for systematic evaluation of "Situated Language Understanding Agents" they propose implementing a framework for implementing rules to be played in "self-play" |
| Outcome: | The proposed model can be evaluated in game-like settings, the authors show . they show that the model can follow game-play instructions and perform better than existing models . |
Sharing the Cost of Success: A Game for Evaluating and Learning Collaborative Multi-Agent Instruction Giving and Following Policies (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent advances in natural language processing have led to language model-based systems that do a good job at creating natural dialogue behaviour but are often verbose and brittle. |
| Approach: | They propose a game that requires two players to coordinate on vision and language observations. |
| Outcome: | The proposed game achieves high success rates when bootstrapped with heuristic partner behaviors that implement insights from the analysis of human-human interactions. |
Using Game Play to Investigate Multimodal and Conversational Grounding in Large Multimodal Models (2025.coling-main)
Copied to clipboard
Sherzod Hakimov, Yerkezhan Abdullayeva, Kushal Koshti, Antonia Schmidt, Yan Weiser, Anne Beyer, David Schlangen
| Challenge: | Existing evaluation paradigms for text-only models are largely limited to a limited number of tasks and require little or no data and training cost. |
| Approach: | They propose to use a game-based evaluation paradigm to evaluate multimodal models by a goal-oriented game (self) play. |
| Outcome: | The proposed evaluation paradigm is more efficient than current methods for text-only models and is more cost-effective than existing methods. |
MM-Claims: A Dataset for Multimodal Claim Detection in Social Media (2022.findings-naacl)
Copied to clipboard
Gullal Singh Cheema, Sherzod Hakimov, Abdul Sittar, Eric Müller-Budack, Christian Otto, Ralph Ewerth
| Challenge: | Using image and text, we investigate the role of image and texts in fake news detection . claim detection is a step in fighting misinformation and as a precursor to prioritize potentially false information for fact-checking. |
| Approach: | They propose a dataset that consists of tweets and corresponding images for claim detection . they evaluate strong unimodal and multimodal baselines and analyze drawbacks of current models . |
| Outcome: | The proposed dataset evaluates strong unimodal and multimodal baselines and examines drawbacks of existing models. |
Images in Language Space: Exploring the Suitability of Large Language Models for Vision & Language Tasks (2023.findings-acl)
Copied to clipboard
| Challenge: | Large language models have demonstrated robust performance on various language tasks using zero-shot or few-shot learning paradigms. |
| Approach: | They propose to use open-source, open-access language models to make visual input accessible to the model using separate verbalisation models. |
| Outcome: | The proposed model can handle visual input but also require strong reasoning component. |
Yes, this Way! Learning to Ground Referring Expressions into Actions with Intra-episodic Feedback from Supportive Teachers (2023.findings-acl)
Copied to clipboard
| Challenge: | In collaborative situations, communication is performed as signalling and recognizing, and the ability to act on these signals is crucial for future machine learning models to collaborate and interact with humans naturally. |
| Approach: | They propose to use a referential language game as an example of a collaborative joint activity to evaluate intra-episodic feedback given by a teacher. |
| Outcome: | The proposed model can generalize on aspects of scene complexity and perform better than providing only the initial statement. |
Playpen: An Environment for Exploring Learning From Dialogue Game Feedback (2025.emnlp-main)
Copied to clipboard
Nicola Horst, Davide Mazzaccara, Antonia Schmidt, Michael Sullivan, Filippo Momentè, Luca Franceschetti, Philipp Sadler, Sherzod Hakimov, Alberto Testoni, Raffaella Bernardi, Raquel Fernández, Alexander Koller, Oliver Lemon, David Schlangen, Mario Giulianelli, Alessandro Suglia
| Challenge: | In this paper, we investigate whether Dialogue Games—goal-directed and rule-governed activities driven predominantly by verbal actions—can also serve as a source of feedback signals for learning. |
| Approach: | They introduce Playpen, an environment for off- and online learning through Dialogue Game self-play, and investigate a representative set of post-training methods: supervised fine-tuning, direct alignment and reinforcement learning with Group Relative Policy Optimization. |
| Outcome: | The proposed model improves performance on unseen instances, but negatively impacts other skills, while interactive learning shows balanced improvements without loss of skills. |