Papers by Yunkun Wang
CodeScope: An Execution-based Multilingual Multitask Multidimensional Benchmark for Evaluating LLMs on Code Understanding and Generation (2024.acl-long)
Copied to clipboard
Weixiang Yan, Haitian Liu, Yunkun Wang, Yunzhe Li, Qian Chen, Wen Wang, Tingyu Lin, Weishan Zhao, Li Zhu, Hari Sundaram, Shuiguang Deng
| Challenge: | Existing benchmarks for evaluating the code understanding and generation capacities of Large Language Models are insufficient . existing benchmarks focus on a narrow range of popular programming languages and specific tasks . |
| Approach: | They propose an execution-based, multilingual, multitask evaluation benchmark for LLMs . they evaluate coding performance from three dimensions: length, difficulty, efficiency . |
| Outcome: | The proposed benchmark covers 43 programming languages and eight coding tasks. |
ExploraCoder: Advancing Code Generation for Multiple Unseen APIs via Planning and Chained Exploration (2025.acl-long)
Copied to clipboard
| Challenge: | Large language models face intrinsic limitations in coding with unseen APIs in training corpora. |
| Approach: | They propose a training-free framework that empowers LLMs to invoke multiple unseen APIs in code solution by planning a complex problem into several API invocation subtasks and experimenting with correct API usage at intermediate steps. |
| Outcome: | The proposed framework significantly improves performance for models lacking prior API knowledge, achieving 11.99% over retrieval-based approaches and 17.28% over pretraining-based methods in pass@10. |