Papers by Tingyu Zhu
CodeScope: An Execution-based Multilingual Multitask Multidimensional Benchmark for Evaluating LLMs on Code Understanding and Generation (2024.acl-long)
Copied to clipboard
Weixiang Yan, Haitian Liu, Yunkun Wang, Yunzhe Li, Qian Chen, Wen Wang, Tingyu Lin, Weishan Zhao, Li Zhu, Hari Sundaram, Shuiguang Deng
| Challenge: | Existing benchmarks for evaluating the code understanding and generation capacities of Large Language Models are insufficient . existing benchmarks focus on a narrow range of popular programming languages and specific tasks . |
| Approach: | They propose an execution-based, multilingual, multitask evaluation benchmark for LLMs . they evaluate coding performance from three dimensions: length, difficulty, efficiency . |
| Outcome: | The proposed benchmark covers 43 programming languages and eight coding tasks. |
When Inverse Data Outperforms: Exploring the Pitfalls of Mixed Data in Multi-Stage Fine-Tuning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for o1-level performance focus on unidirectional supervised fine-tuning (SFT), overlooking the intricate interplay between diverse reasoning patterns. |
| Approach: | They construct a reverse reasoning dataset and examine how it is supervised . they find that naively mixing forward and reverse data during SFT weakens the directional distinction . |
| Outcome: | The proposed model improves accuracy by 1.6%–6.8% over a standard model. |
DGPO: Beyond Pairwise Preferences with Directional Consistent Groupwise Optimization (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for directional consistency alignment of large language models are limited . a recent study suggests reverse supervision as a complement to forward reasoning . |
| Approach: | They propose a framework that aggregates supervision signals at the group level and explicitly models direction-aware alignment through multi-candidate comparisons. |
| Outcome: | The proposed framework achieves 3.2% accuracy improvement across five benchmarks and multiple datasets. |