Papers by Yunfei Zhao
DevEval: A Manually-Annotated Code Generation Benchmark Aligned with Real-World Code Repositories (2024.findings-acl)
Copied to clipboard
Jia Li, Ge Li, Yunfei Zhao, Yongmin Li, Huanyu Liu, Hao Zhu, Lecheng Wang, Kaibo Liu, Zheng Fang, Lanshen Wang, Jiazheng Ding, Xuanming Zhang, Yuqi Zhu, Yihong Dong, Zhi Jin, Binhua Li, Fei Huang, Yongbin Li, Bin Gu, Mengfei Yang
| Challenge: | Existing benchmarks are poorly aligned with real-world code repositories and are insufficient to evaluate the coding abilities of Large Language Models (LLMs). |
| Approach: | They propose a repository-level benchmark named DevEval to evaluate LLMs' coding abilities in real-world code repositories. |
| Outcome: | The proposed benchmarks show that the LLMs perform better in real-world code repositories than existing benchmarks. |
Enhancing Speech Large Language Models with Prompt-Aware Mixture of Audio Encoders (2025.emnlp-main)
Copied to clipboard
Weiqiao Shan, Yuang Li, Yuhao Zhang, Yingfeng Luo, Chen Xu, Xiaofeng Zhao, Long Meng, Yunfei Lu, Min Zhang, Hao Yang, Tong Xiao, JingBo Zhu
| Challenge: | Existing work on integrating audio encoders with large language models (LLMs) has focused on semantic understanding tasks, but different tasks may require distinct features that emphasize either semantic or acoustic aspects. |
| Approach: | They propose to use a prompt-aware mixture to enhance the Speech LLM that uses multiple audio encoders to extract different features based on the prompt. |
| Outcome: | The proposed approach outperforms all single-encoder Speech LLMs on ASR, speaker number verification, and AC tasks. |
NovBench: Evaluating Large Language Models on Academic Paper Novelty Assessment (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for evaluating novelty have been proposed, but there is no systematic evaluation of their ability to generate novelty evaluations. |
| Approach: | They propose a benchmark to evaluate large language models’ ability to generate novelty evaluations in support of human peer review. |
| Outcome: | The proposed framework evaluates the quality of LLM-generated novelty evaluations under different prompting strategies. |
Chinese Synesthesia Detection: New Dataset and Models (2022.findings-acl)
Copied to clipboard
| Challenge: | Synesthesia refers to the description of perceptions in one sensory modality through concepts from other modalities. |
| Approach: | They propose a task called synesthesia detection to extract the sensory word of a sentence and predict the original and synesthetic sensory modalities of the corresponding sensory word. |
| Outcome: | The proposed model achieves state-of-the-art on the Chinese synesthesia dataset. |
DoCIA: An Online Document-Level Context Incorporation Agent for Speech Translation (2025.findings-acl)
Copied to clipboard
Xinglin Lyu, Wei Tang, Yuang Li, Xiaofeng Zhao, Ming Zhu, Junhui Li, Yunfei Lu, Min Zhang, Daimeng Wei, Hao Yang, Min Zhang
| Challenge: | Document-level context is crucial for speech translation due to noise from ASR . incorporating document-level contextual information into ST remains a challenge . |
| Approach: | They develop an online framework that integrates document-level context into machine translation . they use document-based modules to integrate document- level context into ST . |
| Outcome: | The proposed framework outperforms baselines in sentence and discourse metrics . it can correct ASR transcription errors and improve translation performance . |
AIR-Bench: Benchmarking Large Audio-Language Models via Generative Comprehension (2024.acl-long)
Copied to clipboard
Qian Yang, Jin Xu, Wenrui Liu, Yunfei Chu, Ziyue Jiang, Xiaohuan Zhou, Yichong Leng, Yuanjun Lv, Zhou Zhao, Chang Zhou, Jingren Zhou
| Challenge: | Existing benchmarks for audio-centric interaction have impeded advancements in this field . AIR-Bench evaluates LALMs' ability to understand audio signals and interact with humans . |
| Approach: | They propose a benchmark to evaluate the ability of large audio-language models to understand audio signals . they use 19 tasks with approximately 19k single-choice questions to examine single-task ability . |
| Outcome: | The proposed framework evaluates the ability of large audio-language models to understand audio signals and interact with humans in the textual format. |
Rethinking Positional Encoding in Tree Transformer for Code Representation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent works have proposed novel tree Transformers to capture the syntactic structure in source code. |
| Approach: | They propose a novel tree Transformer encoding node positions based on a description method for tree structures to incorporate inductive bias into Transformer. |
| Outcome: | The proposed model outperforms baselines on code summarization and completion tasks across two languages, and it is able to perform better on both local and global paradigms. |
Watermarking with Low-Entropy POS-Guided Token Partitioning and Z-Score-Driven Dynamic Bias for Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing watermarking methods reduce the fidelity of semantics in LLMs . |
| Approach: | They propose a low-entropy token partitioning mechanism and z-score-driven dynamic bias mechanism to enhance semantics. |
| Outcome: | The proposed framework improves semantic fidelity and robustness against bias sparsity attacks. |