Papers by Keyuan Cheng
COMPKE: Complex Question Answering under Knowledge Editing (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks for knowledge editing do not accurately evaluate how well models apply knowledge in real-life situations. |
| Approach: | They propose a benchmark to evaluate how well updated models apply new knowledge in real-life situations. |
| Outcome: | The proposed method achieves 39.47 accuracy on GPT-4o-mini but drops significantly to 3.83 on Qwen2.5-3B. |
CODEMENV: Benchmarking Large Language Models on Code Migration (2025.findings-acl)
Copied to clipboard
Keyuan Cheng, Xudong Shen, Yihao Yang, TengyueWang TengyueWang, Yang Cao, Muhammad Asif Ali, Hanbin Wang, Lijie Hu, Di Wang
| Challenge: | Large language models (LLMs) have demonstrated remarkable proficiency in handling a wide range of tasks within the software engineering domain, but their ability to perform code migration—adapting code to different environments—remains underexplored. |
| Approach: | They propose a benchmark to evaluate large language models’ performance in handling code migration tasks. |
| Outcome: | The proposed benchmark comprises 922 data points across 19 Python and Java packages and offers three tasks to systematically evaluate code migration: identifying version-incompatible functions, determining function changes, and adapting code to target environments. |