Papers by Zhen Thai
OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problems (2024.acl-long)
Copied to clipboard
Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, Jie Liu, Lei Qi, Zhiyuan Liu, Maosong Sun
| Challenge: | Large Language Models (LLMs) and Large Multimodal Models have exceeded general human capabilities in various tasks. |
| Approach: | They present an Olympiad-level bilingual multimodal scientific benchmark featuring 8,476 problems from Olympiad level mathematics and physics competitions. |
| Outcome: | The best performing model, GPT-4V, attains an average score of 17.97% on OlympiadBench, with a mere 10.74% in physics, highlighting the benchmark rigor and the intricacy of physical reasoning. |
DecorateLM: Data Engineering through Corpus Rating, Tagging, and Editing with Language Models (2024.emnlp-main)
Copied to clipboard
Ranchi Zhao, Zhen Thai, Yifan Zhang, Shengding Hu, Jie Zhou, Yunqi Ba, Jie Cai, Zhiyuan Liu, Maosong Sun
| Challenge: | Large Language Models (LLMs) are pre-trained on vast datasets composed of billions of tokens harvested from diverse text sources. |
| Approach: | They propose a data engineering method to refine the pretraining corpus through data rating, tagging and editing. |
| Outcome: | The proposed method improves the quality of the pretraining corpus by enhancing 100 billion tokens of the training corpus. |