Papers by Zhiyu Pan
MT-Video-Bench: A Holistic Video Understanding Benchmark for Evaluating Multimodal LLMs in Multi-Turn Dialogues (2026.findings-acl)
Copied to clipboard
Yaning Pan, Qianqian Xie, Guohui Zhang, Zekun Moore Wang, Yongqian Wen, Yuanxing Zhang, Haoxuan Hu, Zhiyu Pan, Yibing Huang, Zhidong Gan, Yonghong Lin, An Ping, Shihao Li, Yanghai Wang, Tianhao Peng, Jiaheng Liu
| Challenge: | Existing evaluation benchmarks for Multimodal Large Language Models (MLLMs) focus on single-turn question answering, overlooking the complexity of multi-turn dialogues in real-world scenarios. |
| Approach: | They propose a video understanding benchmark for MLLMs in multi-turn dialogues that assesses six core competencies that focus on perceptivity and interactivity. |
| Outcome: | The MT-Video-Bench evaluates 1,000 multi-turn dialogues from diverse domains and reveals significant performance discrepancies and limitations in handling multi-turned video dialogues. |
Multimodal Procedural Planning via Dual Text-Image Prompting (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Embodied agents have demonstrated performance in following instructions informed by texts and images . however, the potential of models providing useful guidelines for humans to complete tasks remains underexplored . |
| Approach: | They propose a multimodal procedural planning task that generates paired text-image plans . this task provides more complementary and informative guidance than unimodal plans a . authors propose modality prompting methods that leverage zero-shot reasoning ability . |
| Outcome: | The proposed method improves the interaction in dual modalities and provides more information than unimodal plans. |