Papers by Jacqueline Staub
Program Synthesis Benchmark for Visual Programming in XLogoOnline Environment (2025.acl-long)
Copied to clipboard
| Challenge: | Large language and multimodal models have shown remarkable success on various benchmarks focused on specific skills such as general-purpose programming, math word problem-solving, and visual question answering. |
| Approach: | They propose a program synthesis benchmark based on real-world programming tasks . they propose 'fine-tuning pipeline' to boost performance of large language models . |
| Outcome: | The proposed model outperforms existing models on tasks that require a combination of skills on visual programming and programming. |
TURTLEAI: Benchmarking Multimodal Models for Visual Programming in Turtle Graphics (2026.findings-acl)
Copied to clipboard
| Challenge: | Vision-language models have been explored for visual programming, but performance is unclear . most prior work focuses on visual programming for productivity . |
| Approach: | They propose a visual programming benchmark that uses visual programming to evaluate VLMs. |
| Outcome: | The proposed model improves on GPT-5, GPT-4o, and Qwen2-VL-72B on real-world tasks by 20% . the proposed model is based on 823 visual programming tasks in the Turtle Graphics domain . |