Papers by Zaiyuan Wang
Code Reffix: A Benchmark for Reflection-Guided Code Repair with Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks focus on the repair generation capability of LLMs, lacking fine-grained evaluation of reflection. |
| Approach: | They propose a benchmark with oracle reflections and a dual-task protocol to decouple evaluation of reflection from repair. |
| Outcome: | The proposed benchmarks show that underperforming reflection capabilities remain a bottleneck for code repair. |
ToolHop: A Query-Driven Benchmark for Evaluating Large Language Models in Multi-Hop Tool Use (2025.acl-long)
Copied to clipboard
Junjie Ye, Zhengyin Du, Xuesong Yao, Weijian Lin, Yufei Xu, Zehui Chen, Zaiyuan Wang, Sining Zhu, Zhiheng Xi, Siyu Yuan, Tao Gui, Qi Zhang, Xuanjing Huang, Jiecao Chen
| Challenge: | Effective evaluation of multi-hop tool use is critical for analyzing the understanding, reasoning, and function-calling capabilities of large language models. |
| Approach: | They propose a dataset that provides rigorous evaluation of multi-hop tool use. |
| Outcome: | The proposed model achieves 49.04% accuracy across five model families. |