Papers by Antonia Schmidt
Using Game Play to Investigate Multimodal and Conversational Grounding in Large Multimodal Models (2025.coling-main)
Copied to clipboard
Sherzod Hakimov, Yerkezhan Abdullayeva, Kushal Koshti, Antonia Schmidt, Yan Weiser, Anne Beyer, David Schlangen
| Challenge: | Existing evaluation paradigms for text-only models are largely limited to a limited number of tasks and require little or no data and training cost. |
| Approach: | They propose to use a game-based evaluation paradigm to evaluate multimodal models by a goal-oriented game (self) play. |
| Outcome: | The proposed evaluation paradigm is more efficient than current methods for text-only models and is more cost-effective than existing methods. |
Playpen: An Environment for Exploring Learning From Dialogue Game Feedback (2025.emnlp-main)
Copied to clipboard
Nicola Horst, Davide Mazzaccara, Antonia Schmidt, Michael Sullivan, Filippo Momentè, Luca Franceschetti, Philipp Sadler, Sherzod Hakimov, Alberto Testoni, Raffaella Bernardi, Raquel Fernández, Alexander Koller, Oliver Lemon, David Schlangen, Mario Giulianelli, Alessandro Suglia
| Challenge: | In this paper, we investigate whether Dialogue Games—goal-directed and rule-governed activities driven predominantly by verbal actions—can also serve as a source of feedback signals for learning. |
| Approach: | They introduce Playpen, an environment for off- and online learning through Dialogue Game self-play, and investigate a representative set of post-training methods: supervised fine-tuning, direct alignment and reinforcement learning with Group Relative Policy Optimization. |
| Outcome: | The proposed model improves performance on unseen instances, but negatively impacts other skills, while interactive learning shows balanced improvements without loss of skills. |