Papers by Yunzhe Li
CodeScope: An Execution-based Multilingual Multitask Multidimensional Benchmark for Evaluating LLMs on Code Understanding and Generation (2024.acl-long)
Copied to clipboard
Weixiang Yan, Haitian Liu, Yunkun Wang, Yunzhe Li, Qian Chen, Wen Wang, Tingyu Lin, Weishan Zhao, Li Zhu, Hari Sundaram, Shuiguang Deng
| Challenge: | Existing benchmarks for evaluating the code understanding and generation capacities of Large Language Models are insufficient . existing benchmarks focus on a narrow range of popular programming languages and specific tasks . |
| Approach: | They propose an execution-based, multilingual, multitask evaluation benchmark for LLMs . they evaluate coding performance from three dimensions: length, difficulty, efficiency . |
| Outcome: | The proposed benchmark covers 43 programming languages and eight coding tasks. |
Advancing Precise Outline-Conditioned Text Generation with Task Duality and Explicit Outline Control (2024.eacl-long)
Copied to clipboard
| Challenge: | Existing studies on outline-conditioned text generation focus on generating text using provided outlines as rough sketches, but lack of clarity and rationality of the rough outlines hampers quality of the generated text. |
| Approach: | They propose a novel task that requires generating stories based on specific, sentence-level outlines. |
| Outcome: | The proposed framework improves the quality of precise outline-conditioned text generation. |
Learning to Instruct: Fine-Tuning a Task-Aware Instruction Optimizer for Black-Box LLMs (2025.findings-emnlp)
Copied to clipboard
Yunzhe Qi, Jinjin Tian, Tianci Liu, Ruirui Li, Tianxin Wei, Hui Liu, Xianfeng Tang, Monica Xiao Cheng, Jingrui He
| Challenge: | Learning to Instruct is a new paradigm for black-box LLMs with inaccessible internal states. |
| Approach: | They propose a new paradigm that formulates instruction optimization as an LLM fine-tuning objective for a white-box “instruction engineer” LLM. |
| Outcome: | The proposed framework outperforms strong baselines in performance and efficiency. |
CodeTransOcean: A Comprehensive Multilingual Benchmark for Code Translation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing code translation datasets focus on a single pair of programming languages . early software systems are developed using programming languages such as Fortran and COBOL . |
| Approach: | They propose a large-scale comprehensive benchmark that supports the largest variety of programming languages for code translation. |
| Outcome: | The proposed framework supports translations between multiple programming languages and a cross-framework dataset for deep learning code across different frameworks. |