Papers by Julius Mayer
Can you SPLICE it together? A Human Curated Benchmark for Probing Visual Reasoning in VLMs (2025.findings-emnlp)
Copied to clipboard
Mohamad Ballout, Okajevo Wilfred, Seyedalireza Yaghoubi, Nohayr Muhammad Abdelmoneim, Julius Mayer, Elia Bruni
| Challenge: | SPLICE is a benchmark designed to probe event-based reasoning across multiple dimensions. |
| Approach: | They introduce a human-curated benchmark to probe event-based reasoning across multiple dimensions. |
| Outcome: | The proposed benchmark includes 3,381 human-filtered videos spanning 12 categories and 180 sub-categories . results show that state-of-the-art vision-language models struggle to match human performance . |
iVISPAR — An Interactive Visual-Spatial Reasoning Benchmark for VLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Vision-Language Models (VLMs) struggle with spatial reasoning and visual alignment, despite their performance on 2D tasks. |
| Approach: | They propose a multimodal benchmark to evaluate VLMs' spatial reasoning capabilities based on the sliding tile puzzle . |
| Outcome: | The proposed model performs better on 2D tasks compared to 3D or text-based settings, but struggles with complex spatial configurations and consistently falls short of human performance. |