Papers by Zeyu Zheng
Mechanistic Interpretability Should Prioritize Feature Consistency in Sparse Autoencoders (2026.acl-long)
Copied to clipboard
Xiangchen Song, Aashiq Muhamed, Yujia Zheng, Lingjing Kong, Zeyu Tang, Mona T. Diab, Virginia Smith, Kun Zhang
| Challenge: | Sparse Autoencoders (SAEs) are a tool in mechanistic interpretability (MI) but the aspiration to identify a canonical set of features is challenged by the observed inconsistency of learned SAE features across different training runs. |
| Approach: | They propose to use the Pairwise Dictionary Mean Correlation Coefficient to quantify SAE feature consistency as an evaluation axis alongside reconstruction and sparsity. |
| Outcome: | The proposed measure is based on the pairwise dictionary mean correlation coefficient (PW-MCC) on LLM activations. |
PaddleSpeech: An Easy-to-Use All-in-One Speech Toolkit (2022.naacl-demo)
Copied to clipboard
Hui Zhang, Tian Yuan, Junkun Chen, Xintong Li, Renjie Zheng, Yuxin Huang, Xiaojie Chen, Enlei Gong, Zeyu Chen, Xiaoguang Hu, Dianhai Yu, Yanjun Ma, Liang Huang
| Challenge: | PaddleSpeech is an open-source speech toolkit that supports speech-to-text and text-to speech tasks. |
| Approach: | They describe the design philosophy and core architecture of PaddleSpeech to support several essential speech-to-text and text-to speech tasks. |
| Outcome: | The proposed framework achieves competitive or state-of-the-art performance on various speech datasets and implements the most popular methods. |
Towards Hierarchical Multi-Step Reward Models for Enhanced Reasoning in Large Language Models (2026.findings-acl)
Copied to clipboard
Teng Wang, Jiang Zhangyi, Zhenqi He, Hailei Gong, Shenyang Tong, Wenhan Yang, Zeyu Li, Yanan Zheng, Zifan He, Zewen Ye, Shengjie Ma, Jianping Zhang
| Challenge: | Existing Process Reward Models (PRMs) are vulnerable to reward hacking and require expensive, large-scale annotation of reasoning steps. |
| Approach: | They propose a reward model approach which evaluates both individual and consecutive reasoning steps from fine-grained and coarse-grounded level. |
| Outcome: | Empirical results show that the proposed model performs better than existing PRMs and is more robust than existing models. |
Demons in the Detail: On Implementing Load Balancing Loss for Training Specialized Mixture-of-Expert Models (2025.acl-long)
Copied to clipboard
Zihan Qiu, Zeyu Huang, Bo Zheng, Kaiyue Wen, Zekun Wang, Rui Men, Ivan Titov, Dayiheng Liu, Jingren Zhou, Junyang Lin
| Challenge: | Existing Mixture-of-Experts training frameworks use a micro-batch to calculate LBL . micro-batches are restricted to a single sequence, preventing expert specialization . |
| Approach: | They propose to use a global-batch to loosen the load balance constraint for MoEs models . they propose to synchronize fi across micro-batches and then use it to calculate the LBL . |
| Outcome: | The proposed global-batch LBL improves the domain specialization of experts . the micro-battery LBL is almost at the sequence level, and the router is pushed to distribute the token evenly . |
Grammar-Based Code Representation: Is It a Worthy Pursuit for LLMs? (2025.findings-acl)
Copied to clipboard
Qingyuan Liang, Zhao Zhang, Zeyu Sun, Zheng Lin, Qi Luo, Xiao Yueyi, Yizhou Chen, Yuqun Zhang, Haotian Zhang, Lu Zhang, Chenbin Chenbin, Yingfei Xiong
| Challenge: | Existing research demonstrates the effectiveness of grammar-based code representations in small-scale models, showing their ability to reduce syntax errors and enhance performance. |
| Approach: | They develop a series of billion-scale grammar-based code representations that incorporate grammar rules into the code generation process. |
| Outcome: | Experiments on HumanEval and MBPP show that grammar-based representations reduce syntax errors and improve performance even in billion-scale models. |
CypherSmith: Transforming Text-to-Cypher Generation for LLMs with Synthetic Data (2026.acl-long)
Copied to clipboard
| Challenge: | Existing datasets are small, domain-limited, and lack diversity, constraining LLM progress. |
| Approach: | They propose a knowledge Graph retrieval tool that can translate natural language questions into structured queries. |
| Outcome: | Extensive experiments show that CypherSmith achieves state-of-the-art performance. |
A Comprehensive Survey of Process Reward Models: Data Generation, Model Construction, and Usage (2026.acl-long)
Copied to clipboard
Congmin Zheng, Jiachen Zhu, Zhuoying Ou, Yuxiang Chen, Kangning Zhang, Rong Shan, Zeyu Zheng, Mengyue Yang, Jianghao Lin, Yong Yu, Weinan Zhang
| Challenge: | Large Language Models (LLMs) have advanced reasoning ability, yet conventional alignment remains dominated by outcome reward models that judge only final answers. |
| Approach: | They summarize applications across math, code, text, multimodal reasoning, robotics, and agents . goal is to clarify design spaces, reveal open challenges, and guide future research toward fine-grained, robust reasoning alignment. |
| Outcome: | The proposed model enables finer credit assignment, richer diagnostics, and improved robustness. |