Papers by Xirui Jiang
Beyond Screenshots: Evaluating VLMs’ Understanding of UI Animations (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent studies of Vision Language Models (VLMs) for UI understanding have focused primarily on static screenshots, leaving it unclear how well these models handle dynamic UI animations. |
| Approach: | They evaluate UI animation models' ability to perceive animation effects and interpret animation meaning . they use motion, context, and perceptual cues to probe factors affecting VLM performance . |
| Outcome: | The proposed model can detect primitive motion, but its interpretation is inconsistent . the proposed model is based on 300 annotated UI animation videos . |