Papers by Qinzheng Sun
MMLU-CF: A Contamination-free Multi-task Language Understanding Benchmark (2025.acl-long)
Copied to clipboard
Qihao Zhao, Yangyu Huang, Tengchao Lv, Lei Cui, Qinzheng Sun, Shaoguang Mao, Xin Zhang, Ying Xin, Qiufeng Yin, Scarlett Li, Furu Wei
| Challenge: | Multiple-choice question datasets like Massive Multitask Language Understanding (MMLU) have inevitably led to benchmark contamination, resulting in unreliable evaluation. |
| Approach: | They propose a contamination-free MCQ benchmark called MMLU-CF which reassesses LLMs’ understanding of world knowledge by averting both unintentional and malicious data contamination. |
| Outcome: | The proposed MMLU-CF reassesses LLMs’ understanding of world knowledge by averting both unintentional and malicious data contamination. |