Papers by Zhenyu He
Dialog-Post: Multi-Level Self-Supervised Objectives and Hierarchical Model for Dialogue Post-Training (2023.acl-long)
Copied to clipboard
| Challenge: | a new method for dialogue representation and understanding is proposed . pre-trained language models (PLMs) are inappropriate for dialogue understanding tasks . |
| Approach: | They propose a method that trains pre-trained language models to fit dialogues . they use a hierarchical segment-wise self-attention network to model dialogues more comprehensively . |
| Outcome: | The proposed method outperforms existing models and achieves a 3.3% improvement on average. |
Label Anchored Contrastive Learning for Language Understanding (2022.naacl-main)
Copied to clipboard
| Challenge: | a novel approach to contrastive learning for language understanding is not fully explored . contrastive training has been widely applied to self-supervised representation learning . |
| Approach: | They propose a label anchored contrastive learning approach for language understanding using a class label. |
| Outcome: | The proposed approach improves on GLUE and CLUE benchmarks by 4.1% compared to the state-of-the-art approaches . the proposed approach also improves under the few-shot and data imbalance settings . |
Hierarchical Acoustic-Semantic Modeling: Modality Separation and Semantic Coherence for Full-Duplex SLMs (2026.acl-long)
Copied to clipboard
Zhenyu Liu, Xuanyu Zhang, Yunxin li, Qixun Teng, Shenyuan Jiang, Haolan Chen, Mingjun Zhao, Fanbo Meng, Yu Xu, Yancheng He, Baotian Hu, Haizhou Li, Min Zhang
| Challenge: | despite significant progress, full-duplex SLMs are constrained by severe modality interference, authors say . modality interferes with acoustic and semantic modeling, making them unintelligent and unnatural . authors propose a hierarchical parameter separation strategy that decouples conflicting modalities in deep layers . |
| Approach: | They propose a hierarchical parameter separation strategy that decouples conflicting modalities in deep layers while preserving cross-modality coherence via a dedicated semantic alignment channel. |
| Outcome: | The proposed method significantly advances the state of the art on full-duplex benchmarks . it decouples conflicting modalities in deep layers while preserving cross-modality coherence . |
Enhancing Auto-regressive Chain-of-Thought through Loop-Aligned Reasoning (2026.eacl-long)
Copied to clipboard
| Challenge: | Chain-of-Thought prompting is a powerful technique for enhancing language model’s reasoning capabilities, but generating long and correct CoT trajectories is challenging. |
| Approach: | They propose to align the steps of Chain-of-Thought reasoning with loop iterations and apply intermediate supervision during the training of Looped Transformers. |
| Outcome: | The proposed method generates accurate reasoning chains for complex problems exceeding training length, and improves performance of the auto-regressive model. |
DKME: Rethinking Coupled Knowledge Memory for Lifelong Model Editing of Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing memory-based editors suffer from catastrophic forgetting as edits accumulate. |
| Approach: | They propose a method which injects factual updates into large language models without retraining or finetuning into existing memory-based editors. |
| Outcome: | Experiments on HalluEditBench, CKnowEdit, and WikiDatacounterfact show that the proposed model achieves a more favorable trade-off between editing success and locality compared to baselines while maintaining more stable performance as the edit scale increases. |
Light-R1: Curriculum SFT, DPO and RL for Long COT from Scratch and Beyond (2025.acl-industry)
Copied to clipboard
Liang Wen, Yunke Cai, Fenrui Xiao, Xin He, Qi An, Zhenyu Duan, Yimin Du, Junchen Liu, Tanglifu Tanglifu, Xiaowei Lv, Haosheng Zou, Yongchao Deng, Shousheng Jia, Xiangzheng Zhang
| Challenge: | Experimental results show that opensource curriculum training is more effective when distinct datasets are available for different training stages. |
| Approach: | They propose an opensource suite for training long reasoning models using publicdata and models. |
| Outcome: | The proposed model outperforms DeepSeek-R1-DistillQwen-32B models in math reasoning. |
REST: Retrieval-Based Speculative Decoding (2024.naacl-long)
Copied to clipboard
| Challenge: | Retrieval-based speculative decoding (REST) is a new language model generation algorithm . it uses existing knowledge to generate draft tokens, allowing for seamless integration and acceleration of any language model. |
| Approach: | They propose a new algorithm that uses a draft language model to generate tokens from existing knowledge. |
| Outcome: | The proposed method achieves a speedup of 1.62 to 2.36 on code or text generation. |
SWE-Swiss: A Multi-Task Fine-Tuning and RL Recipe for High-Performance Issue Resolution (2026.findings-acl)
Copied to clipboard
Zhenyu He, Qingping Yang, Wei Shen, Xiaojian Zhong, Kechi Zhang, Chenxin An, Wenlei Shi, Tianle Cai, Di He, Jiaze Chen, Jingjing Xu
| Challenge: | SWE-Swiss-32B demonstrates strong generalization to other common LLM benchmarks. |
| Approach: | They propose a two-phase training recipe that decomposes issue resolution into three core skills: Localization, Repair, and Unit Test Generation. |
| Outcome: | The proposed model achieves a 60.2% score on the SWE-bench Verified benchmark and is in the top-tier performance bracket of much larger models. |
UniMoE-Audio: Unified Speech and Music Generation with Dynamic-Capacity Mixture-of-Experts (2026.acl-long)
Copied to clipboard
Zhenyu Liu, Yunxin li, Xuanyu Zhang, Qixun Teng, Shenyuan Jiang, Xinyu Chen, Haoyuan Shi, Haolan Chen, Fanbo Meng, Mingjun Zhao, Yu Xu, Yancheng He, Baotian Hu, Haizhou Li, Min Zhang
| Challenge: | Recent advances in unified multimodal models indicate a clear trend towards comprehensive content generation. |
| Approach: | They propose a unified speech and music generation model built upon a novel framework . they propose specialized MoE architectures and curated training strategies to tackle data imbalances . |
| Outcome: | The proposed model achieves state-of-the-art performance on major speech and music generation benchmarks. |
OS-Genesis: Automating GUI Agent Trajectory Construction via Reverse Task Synthesis (2025.acl-long)
Copied to clipboard
Qiushi Sun, Kanzhi Cheng, Zichen Ding, Chuanyang Jin, Yian Wang, Fangzhi Xu, Zhenyu Wu, Chengyou Jia, Liheng Chen, Zhoumianze Liu, Ben Kao, Guohao Li, Junxian He, Yu Qiao, Zhiyong Wu
| Challenge: | Graphical User Interface (GUI) agents powered by Vision-Language Models (VLMs) have demonstrated human-like computer control capability. |
| Approach: | They propose a GUI data synthesis pipeline that reverse engineers GUI trajectory construction process by executing pre-defined tasks. |
| Outcome: | The proposed GUI data synthesis pipeline overcomes the bottlenecks of previous methods that rely on pre-defined tasks and limited data diversity. |
Ted-Tok: Maintaining an Evolving Vocabulary for Lifelong Learning (2026.acl-long)
Copied to clipboard
| Challenge: | a static tokenizer fragments newly emerging lexical items as language evolves . as language grows, a dynamic tokenizer reduces compression efficiency and performance . |
| Approach: | They propose a Temporal Drift Tokenizer that maintains an evolving vocabulary that adapts to emerging linguistic patterns over time. |
| Outcome: | The proposed tokenizer maintains an evolving vocabulary that adapts to emerging linguistic patterns over time. |
Multimodal Neural Machine Translation: A Survey of the State of the Art (2025.emnlp-main)
Copied to clipboard
| Challenge: | Multimodal neural machine translation (MNMT) is a task that aims to translate text into the target language using neural networks. |
| Approach: | They propose to integrate other modalities with textual data to enhance translation performance. |
| Outcome: | The proposed task aims to integrate visual modality with textual data to improve translation quality. |