Papers by Jing-Shu Zheng
FlagEvalMM: A Flexible Framework for Comprehensive Multimodal Model Evaluation (2025.acl-demo)
Copied to clipboard
| Challenge: | FlagEvalMM is an evaluation framework designed to assess multimodal models . it is designed to be used for vision-language understanding and generation tasks . |
| Approach: | They propose an evaluation framework that decouples model inference from evaluation through an independent evaluation service. |
| Outcome: | The evaluation framework offers accurate and efficient insights into model strengths and limitations. |
FlagEval-Arena: A Side-by-Side Comparative Evaluation Platform for Large Language Models and Text-Driven AIGC (2025.acl-demo)
Copied to clipboard
Jing-Shu Zheng, Richeng Xuan, Bowen Qin, Zheqi He, Tongshuai.ren Tongshuai.ren, Xuejing Li, Jin-Ge Yao, Xi Yang
| Challenge: | a new evaluation platform for large language models and text-driven AIGCs is available for free. |
| Approach: | They propose an evaluation platform for side-by-side comparisons of large language models and text-driven AIGC systems. |
| Outcome: | a new evaluation platform for large language models and text-driven AIGC systems is available for free . the platform is more focused on the Chinese language and more models developed by Chinese institutes . |