Papers by Zhiyuan He

Novel Slot Detection: A Benchmark for Discovering Unknown Slot Types in the Task-Oriented Dialogue System (2021.acl-long)

Challenge:	Recent approaches to optimize communication topology rely on single-sample policy gradients with absolute rewards.
Approach:	They propose a topology optimization framework that integrates Group Relative Policy Optimization.
Outcome:	The proposed topology optimization framework outperforms state-of-the-art methods on reasoning and code generation benchmarks.

Copied to clipboard

Yanan Wu, Zhiyuan Zeng, Keqing He, Hong Xu, Yuanmeng Yan, Huixing Jiang, Weiran Xu

Natural Language Interfaces

Challenge:	Existing slot filling models can only recognize pre-defined in-domain slot types from a limited slot set.
Approach:	They introduce a task, Novel Slot Detection, in the task-oriented dialogue system.
Outcome:	The proposed task is based on two public NSD datasets and proposes strong baselines . it aims to identify a sequence of tokens and extract semantic constituents from user queries .

Disentangled Knowledge Transfer for OOD Intent Discovery with Unified Contrastive Learning (2022.acl-short)

Copied to clipboard

Yutao Mou, Keqing He, Yanan Wu, Zhiyuan Zeng, Hong Xu, Huixing Jiang, Wei Wu, Weiran Xu

Enhancing Automated Essay Scoring Performance via Fine-tuning Pre-trained Language Models with Combination of Regression and Ranking (2020.findings-emnlp)

Challenge:	Existing methods to find out out-of-domain (OOD) intents do not take prior knowledge of in-domain data into account.
Approach:	They propose a disentangled knowledge transfer method to bridge the gap between IND pre-training and OOD clustering by using a unified multi-head contrastive learning framework.
Outcome:	The proposed method is able to group new unknown intents into different clusters, enabling future development of the system.

Copied to clipboard

Ruosong Yang, Jiannong Cao, Zhiyuan Wen, Youzheng Wu, Xiaodong He

MMBoundary: Advancing MLLM Knowledge Boundary Awareness through Reasoning Step Confidence Calibration (2025.acl-long)

Challenge:	Recent work on sentence prediction tasks uses shallow neural networks to learn essay representations and constrain calculated scores with regression loss or ranking loss.
Approach:	They propose to use a pre-trained language model to learn text representations first and then to constrain the scores with regression loss or ranking loss.
Outcome:	The proposed model outperforms state-of-the-art models on the Automated Student Assessment Prize dataset.

Copied to clipboard

Zhitao He, Sandeep Polisetty, Zhiyuan Fan, Yuchen Huang, Shujin Wu, Yi R. Fung

Tell Me More! Towards Implicit User Intention Understanding of Language Model Driven Agents (2024.acl-long)

Challenge:	Existing methods calibrate model confidence on entire response, which leads to incorrect answers with high confidence.
Approach:	They propose a framework that advances the knowledge boundary awareness of multimodal large language models through reasoning step confidence calibration.
Outcome:	Empirical results show that the proposed framework outperforms existing methods across domains and metrics.

Copied to clipboard

Cheng Qian, Bingxiang He, Zhong Zhuang, Jia Deng, Yujia Qin, Xin Cong, Zhong Zhang, Jie Zhou, Yankai Lin, Zhiyuan Liu, Maosong Sun

MinerU2.5: A Decoupled Vision-Language Model for Efficient High-Resolution Document Parsing (2026.acl-industry)

Challenge:	Current language model-driven agents lack mechanisms for effective user participation, which is crucial given the vagueness commonly found in user instructions.
Approach:	They propose a benchmark to inspect users’ implicit intentions through explicit queries and a model expert as the upstream in agent design to enhance user-agent interaction.
Outcome:	The proposed approach excels at identifying vague user tasks, recovering and summarizing critical missing information, setting precise and necessary agent execution goals, and minimizing redundant tool usage, thus boosting overall efficiency.

Copied to clipboard

Junbo Niu, Zheng Liu, Zhuangcheng Gu, Bin Wang, Linke Ouyang, Zhiyuan Zhao, Tao Chu, Tianyao He, Fan Wu, Qintong Zhang, Zhenjiang Jin, Guang Liang, Rui Zhang, Wenzheng Zhang, Yuan Qu, Zhifei Ren, Yuefeng Sun, Zirui Tang, Boyu Niu, Yuanhong Zheng, Dongsheng Ma, Ziyang Miao, Hejun Dong, Siyi Qian, Junyuan Zhang, Fangdong Wang, Jingzhou Chen, Xiaomeng Zhao, Liqun Wei, Wei Li, Shasha Wang, RuiLiang Xu, Yuanyuan Cao, Lu Chen, Qianqian Wu, Huaiyu Gu, Lindong Lu, Dechen Lin, null Shenguanlin, Xuanhe Zhou, Linfeng Zhang, Yuhang Zang, Xiaoyi Dong, Jiaqi Wang, Bo Zhang, Lei Bai, Pei Chu, Weijia Li, Jiang Wu, Lijun Wu, Zhenxiang Li, Guangyu Wang, Zhongying Tu, Chao Xu, Kai Chen, Bowen Zhou, Dahua Lin, Wentao Zhang, Conghui He

Green & Sustainable NLP

Challenge:	Document images are characterized by higher resolutions, denser content, and more complex structural layouts.
Approach:	They propose a 1.2B-parameter document parsing vision-language model that decouples layout analysis from local content recognition.
Outcome:	The proposed model surpasses general-purpose and domain-specific models on multiple benchmarks while maintaining significantly lower computational overhead.

HeteroCache: A Dynamic Retrieval Approach to Heterogeneous KV Cache Compression for Long-Context LLM Inference (2026.acl-long)

Copied to clipboard

Zhiyuan Shi, Qibo Qiu, null Xuefeng, Zhonglin Jiang, Li Yu, Jian Jiang, Xiaofei He, Wenxiao Wang

Information Retrieval

Challenge:	Existing static compression methods suffer from coarse-grained caching and high I/O overhead.
Approach:	They propose a training-free dynamic compression framework that uses a sparse attention mechanism to categorize attention heads based on stability and similarity.
Outcome:	The proposed framework achieves state-of-the-art performance on long-context benchmarks and accelerates decoding by up to 3 compared to the original model with a 224K context.

Modeling Discriminative Representations for Out-of-Domain Detection with Supervised Contrastive Learning (2021.acl-short)

Copied to clipboard

Zhiyuan Zeng, Keqing He, Yuanmeng Yan, Zijun Liu, Yanan Wu, Hong Xu, Huixing Jiang, Weiran Xu

Be Careful about Poisoned Word Embeddings: Exploring the Vulnerability of the Embedding Layers in NLP Models (2021.naacl-main)

Challenge:	Existing methods of OOD detection only focus on whether a sample is correctly classified . lack of real OOD examples leads to poor prior knowledge about these unknown intents .
Approach:	They propose a supervised contrastive learning objective to minimize intra-class variance . they employ an adversarial augmentation mechanism to obtain pseudo diverse views .
Outcome:	The proposed method minimizes intra-class variance by pulling together in-domain intents belonging to the same class and maximizes inter-class variation by pushing apart samples from different classes.

Copied to clipboard

Wenkai Yang, Lei Li, Zhiyuan Zhang, Xuancheng Ren, Xu Sun, Bin He

Representation Learning

Challenge:	Recent studies reveal a security threat to natural language processing models, called the Backdoor Attack.
Approach:	They propose to hack a model by modifying one single word embedding vector without sacrificing accuracy on clean samples.
Outcome:	The proposed method is more efficient and stealthier on sentiment analysis and sentence-pair classification tasks.

SimPBL: A Multi-Agent Framework for Project-Based Learning (2026.acl-long)

Copied to clipboard

Daniel Zhang-Li, Joy Jia Yin Lim, Binglin Liu, Shangqing Tu, Zijun Yao, Hao Peng, Jifan Yu, Haoxuan Li, Zhanxin Hao, Ye He, Zekun Li, Jiangyi Wang, Lei Hou, Bin Xu, Xin Cong, Zhiyuan Liu, Huiqin Liu, Yu Zhang, Juanzi Li

Challenge:	Existing LLMs provide partial assistance without modeling these roles, and overly comprehensive help can reduce learner autonomy.
Approach:	They propose a multi-agent framework with an orchestrator agent that provides adaptive scaffolding from interaction logs and collaborator agents that support project work through boundary-aware collaboration.
Outcome:	The proposed framework improves learner examination scores by 14% . it is based on a multi-agent framework with an orchestrator agent .

Comprehensive Benchmarking of Long-Form Speech Generation in Diverse Scenarios (2026.findings-acl)

Copied to clipboard

Changhao Pan, Rui Yang, Han Wang, Zhuan Zhou, Xuming He, Wenxiang Guo, Ziyue Jiang, Ruiqi Li, Yu Zhang, Chenyuhao Wen, Ke Lei, Xiang Yin, Jingyu Lu, Zhiyuan Zhu, Zhou Zhao

Efficient Data Learning for Open Information Extraction with Pre-trained Language Models (2023.findings-emnlp)

Challenge:	Existing evaluation benchmarks for long-form speech are limited to limited domains, creating a significant gap with the diverse downstream applications.
Approach:	They propose a benchmark that decomposes "long-form speech quality" into specific, disentangled dimensions.
Outcome:	The proposed benchmark decomposes “long-form speech quality” into specific, disentangled dimensions.

Copied to clipboard

Zhiyuan Fan, Shizhu He

LeanK: Learnable K Cache Channel Pruning for Efficient Decoding (2025.emnlp-main)

Challenge:	Experimental results indicate that, compared to previous SOTA methods, OK-IE requires only 1/100 of the training data (900 instances) and 1/120 of the time (3 minutes) to achieve comparable results.
Approach:	They propose a framework that transforms OpenIE into the pre-training task form of the T5 model, thereby reducing the need for extensive training data.
Outcome:	The proposed framework transforms OpenIE into the pre-training task form of the T5 model, reducing the need for extensive training data and significantly reducing training time.

Copied to clipboard

Yike Zhang, Zhiyuan He, Huiqiang Jiang, Chengruidong Zhang, Yuqing Yang, Jianyong Wang, Lili Qiu

Green & Sustainable NLP

Challenge:	Existing efforts to optimize the key-value (KV) cache include: (1) Eviction, which discards cache of less important tokens; (2) Selection, which retains the full KV cache but selectively reads relevant entries.
Approach:	They propose a learning-based method that prunes unimportant key (K) cache channels by leveraging static channel sparsity.
Outcome:	Experiments show that LeanK reduces GPU memory and accelerates decoding without sacrificing accuracy.

Empowering Reliable Visual-Centric Instruction Following in MLLMs (2026.findings-acl)

Copied to clipboard

Weilei He, Feng Ju, Zhiyuan Fan, Rui Min, Minhao Cheng

The Right Time Matters: Data Arrangement Affects Zero-Shot Generalization in Instruction Tuning (2025.findings-acl)

Challenge:	Existing benchmarks for evaluating instruction-following capabilities focus on verbal instructions in the textual modality.
Approach:	They propose to incorporate vision-dependent constraints into instruction design to enable a more rigorous assessment of how well MLLMs align their outputs with both visual input and textual instructions.
Outcome:	The proposed benchmark incorporates vision-dependent constraints into instruction design, enabling a more rigorous and fine-grained assessment of how well MLLMs align their outputs with both visual input and textual instructions.

Copied to clipboard

Bingxiang He, Ning Ding, Cheng Qian, Jia Deng, Ganqu Cui, Lifan Yuan, Haiwen Hong, Huan-ang Gao, Longtao Huang, Hui Xue, Huimin Chen, Zhiyuan Liu, Maosong Sun

Accelerating Prefilling via Decoding-time Contribution Sparsity (2026.findings-acl)

Challenge:	Existing work on instruction tuning has focused on task level, without considering that tasks are artificially defined and, to LLMs, merely consist of tokens and representations.
Approach:	They propose a training data arrangement framework that allows for continual learning and loss reduction.
Outcome:	The proposed framework promotes continual learning and loss reduction on unseen tasks.

Copied to clipboard

Zhiyuan He, Yike Zhang, Chengruidong Zhang, Huiqiang Jiang, Yuqing Yang, Lili Qiu

Challenge:	Existing acceleration methods exploit attention score sparsity by estimating blocks with high attention scores and applying dynamic sparse attention.
Approach:	They propose a method which replaces dense attention with Triangle attention in a subset of layers to reduce the time needed to decode.
Outcome:	Experiments show that TriangleMix achieves near-lossless performance on long-context and long-constrast reasoning benchmarks while significantly improving efficiency.

OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problems (2024.acl-long)

Copied to clipboard

Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, Jie Liu, Lei Qi, Zhiyuan Liu, Maosong Sun

Position Engineering: Boosting Large Language Models through Positional Information Manipulation (2024.emnlp-main)

Challenge:	Large Language Models (LLMs) and Large Multimodal Models have exceeded general human capabilities in various tasks.
Approach:	They present an Olympiad-level bilingual multimodal scientific benchmark featuring 8,476 problems from Olympiad level mathematics and physics competitions.
Outcome:	The best performing model, GPT-4V, attains an average score of 17.97% on OlympiadBench, with a mere 10.74% in physics, highlighting the benchmark rigor and the intricacy of physical reasoning.

Copied to clipboard

Zhiyuan He, Huiqiang Jiang, Zilong Wang, Yuqing Yang, Luna Qiu, Lili Qiu

Pretrain-KGE: Learning Knowledge Representation from Pretrained Language Models (2020.findings-emnlp)

Challenge:	Recent advances in Large Language Models (LLMs) have demonstrated significant strides towards achieving artificial general intelligence.
Approach:	They propose a technique termed position engineering which alters the positional information in the prompt without modifying the text itself.
Outcome:	The proposed technique significantly improves on the baseline in retrieval-augmented generation and in-context learning scenarios.

Copied to clipboard

Zhiyuan Zhang, Xiaoqian Liu, Yi Zhang, Qi Su, Xu Sun, Bin He

Neural Network Surgery: Injecting Data Patterns into Pre-trained Models with Minimal Instance-wise Side Effects (2021.naacl-main)

Challenge:	Existing knowledge graph embedding models suffer from limited knowledge representation due to sparse and noisy dataset annotations.
Approach:	They propose to use pretrained language models to enhance knowledge representation by leveraging world knowledge from pretrained models.
Outcome:	Extensive experiments show that the proposed framework can improve results over existing models.

Copied to clipboard

Zhiyuan Zhang, Xuancheng Ren, Qi Su, Xu Sun, Bin He

Beat LLMs at Their Own Game: Zero-Shot LLM-Generated Text Detection via Querying ChatGPT (2023.emnlp-main)

Challenge:	Existing neural network tuning methods cause instance-wise side effects . et al., 2018: a new approach to perform neural network surgery .
Approach:	They propose to perform neural network surgery by only changing 10-5 parameters . they propose to use a dynamic selecting method to achieve the best overall performance .
Outcome:	The proposed method achieves the best overall performance and induces fewer instance-wise side effects by changing only 10-5 of the parameters.

Copied to clipboard

Biru Zhu, Lifan Yuan, Ganqu Cui, Yangyi Chen, Chong Fu, Bingxiang He, Yangdong Deng, Zhiyuan Liu, Maosong Sun, Ming Gu

UltraEval: A Lightweight Platform for Flexible and Comprehensive Evaluation for LLMs (2024.acl-demos)

Challenge:	Large language models (LLMs) are capable of performing tasks but are likely to be misused.
Approach:	They propose a zero-shot black-box method to detect LLM-generated texts . they revise the text to be detected using the ChatGPT model .
Outcome:	The proposed method can detect LLM-generated texts with a zero-shot black-box model . it is based on intuition that the model will make fewer revisions to LLMs than to human-written texts .

Copied to clipboard

Chaoqun He, Renjie Luo, Shengding Hu, Ranchi Zhao, Jie Zhou, Hanghao Wu, Jiajie Zhang, Xu Han, Zhiyuan Liu, Maosong Sun

Revisit Overconfidence for OOD Detection: Reassigned Contrastive Learning with Adaptive Class-dependent Threshold (2022.naacl-main)

Challenge:	Existing evaluation platforms are complex and poorly modularized, hindering seamless incorporation into researcher’s workflows.
Approach:	They propose a lightweight evaluation framework characterized by lightweight, comprehensiveness, modularity, and efficiency that integrates models, data, and metrics into a unified evaluation workflow.
Outcome:	The proposed evaluation framework is lightweight, comprehensive, modular, and efficient.

Copied to clipboard

Yanan Wu, Keqing He, Yuanmeng Yan, QiXiang Gao, Zhiyuan Zeng, Fujia Zheng, Lulu Zhao, Huixing Jiang, Wei Wu, Weiran Xu

Mathematical Proof as a Litmus Test: Revealing Failure Modes of Advanced Large Reasoning Models (2026.acl-long)

Challenge:	Existing methods for OOD detection are based on labeled in-domain data . detecting out-of-domain (OOD) or unknown intents is challenging .
Approach:	They propose a novel reassigned contrastive learning method to discriminate IND intents for over-confident OOD and an adaptive class-dependent local threshold mechanism to separate similar IND and OOD intents.
Outcome:	The proposed method is effective for both aspects of overconfidence issues.

Copied to clipboard

Dadi Guo, Jiayu Liu, Zhiyuan Fan, Zhitao He, Haoran Li, Yuxin Li, Yumeng Wang, Yi R. Fung

Reasoning

Challenge:	Large reasoning models have demonstrated remarkable mathematical problem-solving abilities, but their true reasoning shortcomings are often hidden.
Approach:	They propose to leverage the rigor and methodological complexity of mathematical proofs as a diagnostic tool to expose hidden failures.
Outcome:	The proposed model evaluation exploits the rigor and complexity of proof problems to uncover 10 fine-grained errors.

Adversarial Self-Supervised Learning for Out-of-Domain Detection (2021.naacl-main)

Copied to clipboard

Zhiyuan Zeng, Keqing He, Yuanmeng Yan, Hong Xu, Weiran Xu

Responsible & Trustworthy NLP

Challenge:	Existing methods for detecting out-of-domain (OOD) intents are unsupervised and require extensive labeled data.
Approach:	They propose a self-supervised contrastive learning framework to model discriminative semantic features from unlabeled data.
Outcome:	The proposed framework outperforms baseline methods on two public benchmark datasets with a statistically significant margin.

ORMind: A Cognitive-Inspired End-to-End Reasoning Framework for Operations Research (2025.acl-industry)

Copied to clipboard

Zhiyuan Wang, Bokui Chen, Yinya Huang, Qingxing Cao, Ming He, Jianping Fan, Xiaodan Liang

Reasoning

Challenge:	Large Language Models (LLMs) have shown promising results in various domains, but their practical application in industry-relevant operations research presents significant challenges and opportunities.
Approach:	They propose a cognitive-inspired framework that enhances optimization through counterfactual reasoning . they use a workflow that transforms requirements into mathematical models and executable solver code .
Outcome:	Experiments show that ORMind outperforms existing methods in the NL4Opt dataset and ComplexOR dataset.

PRESTO: Progressive Pretraining Enhances Synthetic Chemistry Outcomes (2024.findings-emnlp)

Copied to clipboard

He Cao, Yanjun Shao, Zhiyuan Liu, Zijing Liu, Xiangru Tang, Yuan Yao, Yu Li