Papers by Changhong Jin
SciAssess: Benchmarking LLM Proficiency in Scientific Literature Analysis (2025.findings-naacl)
Copied to clipboard
Hengxing Cai, Xiaochen Cai, Junhan Chang, Sihang Li, Lin Yao, Wang Changxin, Zhifeng Gao, Hongshuai Wang, Li Yongge, Mujie Lin, Shuwen Yang, Jiankun Wang, Mingjun Xu, Jin Huang, Xi Fang, Jiaxi Zhuang, Yuqi Yin, Yaqi Li, Changhong Chen, Zheng Cheng, Zifeng Zhao, Linfeng Zhang, Guolin Ke
| Challenge: | Existing benchmarks fail to adequately evaluate the proficiency of Large Language Models (LLMs) Existing standards do not cover the skills needed to evaluate LLMs in scientific literature analysis. |
| Approach: | They propose a benchmark to evaluate the proficiency of large language models in scientific literature analysis. |
| Outcome: | SciAssess evaluates 11 LLMs on multiple tasks across scientific fields. |
LiveFact: A Dynamic, Time-Aware Benchmark for LLM-Driven Fake News Detection (2026.acl-long)
Copied to clipboard
| Challenge: | Current evaluation frameworks are static and vulnerable to benchmark data contamination . current models are ineffective at assessing reasoning under temporal uncertainty . |
| Approach: | They propose a live-based benchmark that simulates the real-world "fog of war" they propose evaluating models on their ability to reason with evolving, incomplete information . |
| Outcome: | The proposed model outperforms proprietary state-of-the-art models in classification and evidence mode . it also provides a component to monitor BDC explicitly . |
DCR: Quantifying Data Contamination in LLMs Evaluation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) memorize evaluation data during training, inflating performance metrics and undermining genuine generalization assessment. |
| Approach: | They propose a framework to detect and quantify benchmark data contamination (BDC) by synthesizing contamination scores via a fuzzy inference system. |
| Outcome: | The proposed framework detects and quantifies BDC risk across semantic, informational, data, and label levels. |