Papers by Chengkai Liu
DisastIR: A Comprehensive Information Retrieval Benchmark for Disaster Management (2025.findings-emnlp)
Copied to clipboard
Kai Yin, Xiangjue Dong, Chengkai Liu, Lipai Huang, Yiming Xiao, Zhewei Liu, Ali Mostafavi, James Caverlee
| Challenge: | Existing information retrieval benchmarks focus on general or specialized domains, such as medicine or finance, neglecting the unique linguistic complexity and diverse information needs encountered in disaster management scenarios. |
| Approach: | DisastIR is the first comprehensive IR evaluation benchmark specifically tailored for disaster management. |
| Outcome: | DisastIR covers 48 retrieval tasks derived from six search intents and eight general disaster categories . evaluations show no single model excelling universally . |
DisastQA: A Comprehensive Benchmark for Evaluating Question Answering in Disaster Management (2026.findings-acl)
Copied to clipboard
Zhitong Chen, Kai Yin, Xiangjue Dong, Chengkai Liu, Xiangpeng Li, Bo Li, Junwei Ma, Yiming Xiao, Ali Mostafavi, James Caverlee
| Challenge: | Existing benchmarks for question answering (QA) are lacking in a high-stakes environment. |
| Approach: | They propose a rigorously verified benchmark of 3,000 expert-annotated questions . they propose 'keypoint-based evaluation protocol' emphasizing factual completeness over verbosity . |
| Outcome: | Experiments with 20 models reveal substantial divergences from general-purpose models such as MMLU-Pro. |
DMRetriever: A Family of Models for Improved Text Retrieval in Disaster Management (2026.acl-long)
Copied to clipboard
| Challenge: | Existing models fail to handle the varied search intents inherent to disaster management scenarios, resulting in inconsistent and unreliable performance. |
| Approach: | They propose a new series of dense retrieval models tailored for disaster management that train on a three-stage framework with unsupervised contrastive pre-training and difficulty-aware progressive instruction fine-tuning. |
| Outcome: | The proposed model outperforms baseline models by 13.3 times and 33 times over baselines with only 7.6% of their parameters. |