Papers by Zhiyuan Wu
Novel Slot Detection: A Benchmark for Discovering Unknown Slot Types in the Task-Oriented Dialogue System (2021.acl-long)
Copied to clipboard
| Challenge: | Existing slot filling models can only recognize pre-defined in-domain slot types from a limited slot set. |
| Approach: | They introduce a task, Novel Slot Detection, in the task-oriented dialogue system. |
| Outcome: | The proposed task is based on two public NSD datasets and proposes strong baselines . it aims to identify a sequence of tokens and extract semantic constituents from user queries . |
Knowledgeable Prompt-tuning: Incorporating Knowledge into Prompt Verbalizer for Text Classification (2022.acl-long)
Copied to clipboard
| Challenge: | Recent studies suggest that pre-trained language models have gained rich knowledge during pre-training. |
| Approach: | They propose to tune pre-trained language models with task-specific prompts to improve and stabilize prompttuning. |
| Outcome: | Extensive experiments on zero and few-shot text classification tasks show that prompt-tuning improves and stabilizes prompttun-ing. |
AGGC: Adaptive Group Gradient Clipping for Stabilizing Large Language Model Training (2026.findings-acl)
Copied to clipboard
| Challenge: | Adaptive group-wise gradient clipping (AGGC) is a new approach to stabilize training of Large Language Models. |
| Approach: | They propose a method to stabilize gradient clipping by partitioning parameters into groups based on functional types and a time-dependent scheduling mechanism to balance exploration and convergence. |
| Outcome: | The proposed algorithm outperforms standard LoRA and achieves 72.93% accuracy . it can be integrated into existing pipelines with negligible overhead. |
From Adversarial Arms Race to Model-centric Evaluation: Motivating a Unified Automatic Robustness Evaluation Framework (2023.findings-acl)
Copied to clipboard
Yangyi Chen, Hongcheng Gao, Ganqu Cui, Lifan Yuan, Dehan Kong, Hanlu Wu, Ning Shi, Bo Yuan, Longtao Huang, Hui Xue, Zhiyuan Liu, Maosong Sun, Heng Ji
| Challenge: | Existing models of robustness evaluation are incomprehensive, impractical, and invalid . |
| Approach: | They propose a framework for automatic robustness evaluation that shifts towards model-centric evaluation to further exploit the advantages of adversarial attacks. |
| Outcome: | The proposed framework is based on a model-centric evaluation protocol and a robustness evaluation protocol. |
STARS: A Unified Framework for Singing Transcription, Alignment, and Refined Style Annotation (2025.findings-acl)
Copied to clipboard
Wenxiang Guo, Yu Zhang, Changhao Pan, Zhiyuan Zhu, Ruiqi Li, ZheTao Chen, Wenhao Xu, Fei Wu, Zhou Zhao
| Challenge: | Existing automated singing annotation (ASA) methods tackle isolated aspects of the annotation pipeline. |
| Approach: | They propose a framework that addresses transcription, alignment, and refined style annotations. |
| Outcome: | The proposed framework delivers comprehensive multi-level annotations encompassing: (1) precise phoneme-audio alignment, (2) robust note transcription and temporal localization, (3) expressive vocal technique identification, and (4) global stylistic characterization including emotion and pace. |
Encoding and Controlling Global Semantics for Long-form Video Question Answering (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to find answers for long videos fail to reason over the whole sequence of video, leading to sub-optimal performance. |
| Approach: | They propose a state space layer to integrate global semantics into video . they use a gating unit to enable controllability over the flow of global semantic into visual representations. |
| Outcome: | The proposed framework is able to integrate global semantics into visual representations. |
Disentangled Knowledge Transfer for OOD Intent Discovery with Unified Contrastive Learning (2022.acl-short)
Copied to clipboard
| Challenge: | Existing methods to find out out-of-domain (OOD) intents do not take prior knowledge of in-domain data into account. |
| Approach: | They propose a disentangled knowledge transfer method to bridge the gap between IND pre-training and OOD clustering by using a unified multi-head contrastive learning framework. |
| Outcome: | The proposed method is able to group new unknown intents into different clusters, enabling future development of the system. |
RepoAgent: An LLM-Powered Open-Source Framework for Repository-level Code Documentation Generation (2024.emnlp-demo)
Copied to clipboard
Qinyu Luo, Yining Ye, Shihao Liang, Zhong Zhang, Yujia Qin, Yaxi Lu, Yesai Wu, Xin Cong, Yankai Lin, Yingli Zhang, Xiaoyin Che, Zhiyuan Liu, Maosong Sun
| Challenge: | Xia et al., 2018) demonstrate that a large language model can generate and maintain high-quality code documentation. |
| Approach: | They propose a large language model powered open-source framework for generating, maintaining, and updating code documentation. |
| Outcome: | The proposed framework generates high-quality documentation for the entire project. |
Enhancing Automated Essay Scoring Performance via Fine-tuning Pre-trained Language Models with Combination of Regression and Ranking (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work on sentence prediction tasks uses shallow neural networks to learn essay representations and constrain calculated scores with regression loss or ranking loss. |
| Approach: | They propose to use a pre-trained language model to learn text representations first and then to constrain the scores with regression loss or ranking loss. |
| Outcome: | The proposed model outperforms state-of-the-art models on the Automated Student Assessment Prize dataset. |
MMBoundary: Advancing MLLM Knowledge Boundary Awareness through Reasoning Step Confidence Calibration (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods calibrate model confidence on entire response, which leads to incorrect answers with high confidence. |
| Approach: | They propose a framework that advances the knowledge boundary awareness of multimodal large language models through reasoning step confidence calibration. |
| Outcome: | Empirical results show that the proposed framework outperforms existing methods across domains and metrics. |
MinerU2.5: A Decoupled Vision-Language Model for Efficient High-Resolution Document Parsing (2026.acl-industry)
Copied to clipboard
Junbo Niu, Zheng Liu, Zhuangcheng Gu, Bin Wang, Linke Ouyang, Zhiyuan Zhao, Tao Chu, Tianyao He, Fan Wu, Qintong Zhang, Zhenjiang Jin, Guang Liang, Rui Zhang, Wenzheng Zhang, Yuan Qu, Zhifei Ren, Yuefeng Sun, Zirui Tang, Boyu Niu, Yuanhong Zheng, Dongsheng Ma, Ziyang Miao, Hejun Dong, Siyi Qian, Junyuan Zhang, Fangdong Wang, Jingzhou Chen, Xiaomeng Zhao, Liqun Wei, Wei Li, Shasha Wang, RuiLiang Xu, Yuanyuan Cao, Lu Chen, Qianqian Wu, Huaiyu Gu, Lindong Lu, Dechen Lin, null Shenguanlin, Xuanhe Zhou, Linfeng Zhang, Yuhang Zang, Xiaoyi Dong, Jiaqi Wang, Bo Zhang, Lei Bai, Pei Chu, Weijia Li, Jiang Wu, Lijun Wu, Zhenxiang Li, Guangyu Wang, Zhongying Tu, Chao Xu, Kai Chen, Bowen Zhou, Dahua Lin, Wentao Zhang, Conghui He
| Challenge: | Document images are characterized by higher resolutions, denser content, and more complex structural layouts. |
| Approach: | They propose a 1.2B-parameter document parsing vision-language model that decouples layout analysis from local content recognition. |
| Outcome: | The proposed model surpasses general-purpose and domain-specific models on multiple benchmarks while maintaining significantly lower computational overhead. |
AgentCPM-GUI: Building Mobile-Use Agents with Reinforcement Fine-Tuning (2025.emnlp-demos)
Copied to clipboard
Zhong Zhang, Yaxi Lu, Yikun Fu, Yupeng Huo, Shenzhi Yang, Yesai Wu, Han Si, Xin Cong, Haotian Chen, Yankai Lin, Xie Xie, Wei Zhou, Wang Xu, Zhou Su, Zhongwu Zhai, Xiaoming Liu, null Meiyudong, Jianming Xu, Hongyan Tian, Chongyi Wang, Chi Chen, Yuan Yao, Zhiyuan Liu, Maosong Sun
| Challenge: | Large language model agents have enabled GUI-based automation, but their deployment is limited by noisy data, poor generalization, and lack of support for non-English GUIs. |
| Approach: | They propose an 8B-parameter GUI agent built for robust and efficient on-device GUI interaction. |
| Outcome: | The proposed GUI agent achieves promising performance on five public benchmarks and proposed Chinese benchmark CAGUI. |
Deciphering Digital Detectives: Understanding LLM Behaviors and Capabilities in Multi-Agent Mystery Games (2024.findings-acl)
Copied to clipboard
| Challenge: | In this study, we explore the application of Large Language Models (LLMs) in Jubensha, a Chinese detective role-playing game and a novel area in Artificial Intelligence (AI) driven gaming. |
| Approach: | They propose to use large language models to foster AI agent development in Jubensha, a Chinese detective role-playing game. |
| Outcome: | The proposed framework enables AI agents to engage in Jubensha games autonomously. |
MolTC: Towards Molecular Relational Modeling In Language Models (2024.findings-acl)
Copied to clipboard
Junfeng Fang, Shuai Zhang, Chang Wu, Zhengyi Yang, Zhiyuan Liu, Sihang Li, Kun Wang, Wenjie Du, Xiang Wang
| Challenge: | Molecular Relational Learning (MRL) is a promising way to understand interactions between molecular pairs. |
| Approach: | They propose a novel LLM-based multi-modal framework for molecular interaction modeling following Chain-of-Thought (CoT) theory which integrates graphical information of two molecules in pair. |
| Outcome: | The proposed framework integrates graphical information of two molecules in pair. |
Enhancing Long-Chain Reasoning Distillation through Error-Aware Self-Reflection (2026.findings-acl)
Copied to clipboard
Zhuoyang Wu, Xinze Li, Zhenghao Liu, Yukun Yan, Zhiyuan Liu, Minghe Yu, Cheng Yang, Yu Gu, Ge Yu, Maosong Sun
| Challenge: | Existing studies treat SLMs as student models and use long-form Chains-of-Thought (CoTs) as supervision signals for Supervised Fine-Tuning (SFT). Existing research focuses on distilling reasoning ability from LLMs to enhance the mathematical reasoning performance of small-scale models. |
| Approach: | They propose a framework that refines teacher CoTs through an error-aware reflection process to enable the student model to construct more tailored teacher Cots. |
| Outcome: | Experiments on multiple mathematical reasoning benchmarks show that ORION improves performance by more than 2% over all baselines. |
RouteMoA: Dynamic Routing without Pre-Inference Boosts Efficient Mixture-of-Agents (2026.acl-long)
Copied to clipboard
Jize Wang, Han Wu, Zhiyuan You, Yiming Song, Yijun Wang, Zifei Shan, Yining Li, Songyang Zhang, Xinyi Le, Cailian Chen, Xinping Guan, Dacheng Tao
| Challenge: | Existing methods for mixing-of-agents (MoA) lack model selection criteria and struggle with large model pools. |
| Approach: | They propose a mixture-of-agents framework with dynamic routing that uses a lightweight scorer to perform initial screening and refines the model scores through self- and cross-assessment. |
| Outcome: | The proposed framework outperforms existing methods for large model pools and tasks . it reduces cost by 89.8% and latency by 63.6% in the large-scale model pool. |
Modeling Discriminative Representations for Out-of-Domain Detection with Supervised Contrastive Learning (2021.acl-short)
Copied to clipboard
| Challenge: | Existing methods of OOD detection only focus on whether a sample is correctly classified . lack of real OOD examples leads to poor prior knowledge about these unknown intents . |
| Approach: | They propose a supervised contrastive learning objective to minimize intra-class variance . they employ an adversarial augmentation mechanism to obtain pseudo diverse views . |
| Outcome: | The proposed method minimizes intra-class variance by pulling together in-domain intents belonging to the same class and maximizes inter-class variation by pushing apart samples from different classes. |
Dynamic Anticipation and Completion for Multi-Hop Reasoning over Sparse Knowledge Graph (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing reasoning methods for sparse KGs are incomplete and lack of evidential paths to target entities makes multi-hop reasoning difficult. |
| Approach: | They propose a multi-hop reasoning model over sparse KGs to solve this problem . they use latent prediction of embedding-based models to make the model perform more potential path search over sparses . |
| Outcome: | The proposed method outperforms state-of-the-art models on five datasets from Freebase, NELL and Wikidata. |
Open Relation Extraction: Relational Knowledge Transfer from Supervised Data to Unsupervised Data (D19-1)
Copied to clipboard
| Challenge: | Existing methods to extract relational facts from open domain corpora are time-consuming and human-intensive. |
| Approach: | They propose a framework to learn similarity metrics of relations from labeled data . they propose to transfer relational knowledge to identify novel relations in unlabeled data. |
| Outcome: | Experiments on two real-world datasets show that the proposed framework improves compared with state-of-the-art methods. |
Cost-Optimal Grouped-Query Attention for Long-Context Modeling (2025.emnlp-main)
Copied to clipboard
Yingfa Chen, Yutong Wu, Chenyang Song, Zhen Leng Thai, Xingyu Shen, Xu Han, Zhiyuan Liu, Maosong Sun
| Challenge: | Current GQA configurations overlook how context length influences inference cost . |
| Approach: | They propose a recipe for deriving cost-optimal GQA configurations that decouple the total head size from the hidden size and allow more flexible control over attention FLOPs. |
| Outcome: | The proposed configurations reduce memory usage and FLOPs by more than 50% compared to Llama-3's GQA, with *no degradation in model capabilities*. |
Integrating Representation Subspace Mapping with Unimodal Auxiliary Loss for Attention-based Multimodal Emotion Recognition (2024.lrec-main)
Copied to clipboard
Xulong Du, Xingnan Zhang, Dandan Wang, Yingying Xu, Zhiyuan Wu, Shiqing Zhang, Xiaoming Zhao, Jun Yu, Liangliang Lou
| Challenge: | Existing methods to identify emotions rely on a large modality gap in their representations . |
| Approach: | They propose a representation subspace mapping module that maps each modality into two distinct subspaces and a cross-modality attention module that leverages auxiliary loss to remove the noise unrelated to emotion classification. |
| Outcome: | The proposed approach achieves superior performance to state-of-the-art MER methods on the IEMOCAP and MSP-Improv datasets. |
A Survey of Retentive Network (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on the effectiveness of the Retentive Networks have not yet been conducted. |
| Approach: | They propose a retention mechanism that integrates the inductive bias of recurrent neural networks with the parallelizable training advantages of attention-based models. |
| Outcome: | The proposed retention mechanism combines the inductive bias of recurrent neural networks with the parallelizable training advantages of attention-based models. |
UltraEval: A Lightweight Platform for Flexible and Comprehensive Evaluation for LLMs (2024.acl-demos)
Copied to clipboard
Chaoqun He, Renjie Luo, Shengding Hu, Ranchi Zhao, Jie Zhou, Hanghao Wu, Jiajie Zhang, Xu Han, Zhiyuan Liu, Maosong Sun
| Challenge: | Existing evaluation platforms are complex and poorly modularized, hindering seamless incorporation into researcher’s workflows. |
| Approach: | They propose a lightweight evaluation framework characterized by lightweight, comprehensiveness, modularity, and efficiency that integrates models, data, and metrics into a unified evaluation workflow. |
| Outcome: | The proposed evaluation framework is lightweight, comprehensive, modular, and efficient. |
Revisit Overconfidence for OOD Detection: Reassigned Contrastive Learning with Adaptive Class-dependent Threshold (2022.naacl-main)
Copied to clipboard
Yanan Wu, Keqing He, Yuanmeng Yan, QiXiang Gao, Zhiyuan Zeng, Fujia Zheng, Lulu Zhao, Huixing Jiang, Wei Wu, Weiran Xu
| Challenge: | Existing methods for OOD detection are based on labeled in-domain data . detecting out-of-domain (OOD) or unknown intents is challenging . |
| Approach: | They propose a novel reassigned contrastive learning method to discriminate IND intents for over-confident OOD and an adaptive class-dependent local threshold mechanism to separate similar IND and OOD intents. |
| Outcome: | The proposed method is effective for both aspects of overconfidence issues. |
Learning to Generate Structured Output with Schema Reinforcement Learning (2025.acl-long)
Copied to clipboard
Yaxi Lu, Haolun Li, Xin Cong, Zhong Zhang, Yesai Wu, Yankai Lin, Zhiyuan Liu, Fangming Liu, Maosong Sun
| Challenge: | Recent advances in large language models have facilitated the development of intelligent applications like automatic web search (Qin et al., 2023) Several methods exist for generating JSON strings from LLMs, including Prompting but often miss certain schemas. |
| Approach: | They propose to use 40K different JSON schemas to assess models' ability to generate valid JSON outputs. |
| Outcome: | The proposed model improves both in generating JSON outputs and downstream tasks. |
TSPO: Breaking the Double Homogenization Dilemma in Multi-turn Search Policy Optimization (2026.findings-acl)
Copied to clipboard
Shichao Ma, Zhiyuan Ma, Ming Yang, Xiaofan Li, Xing Wu, Jintao Du, Yu Cheng, Weiqiang Wang, Qiliang Liu, Zhengyang Zhou, Yang Wang
| Challenge: | Large Language Models (LLMs) can solve complex tasks through iterative information retrieval. |
| Approach: | They propose a turn-level stage-aware policy optimization approach to solve this problem . they introduce a first-occurrence latent reward mechanism to allocate partial rewards . |
| Outcome: | Experiments show that TSPO outperforms state-of-the-art models on Qwen2.5-3B and 7B models. |
DebugBench: Evaluating Debugging Capability of Large Language Models (2024.findings-acl)
Copied to clipboard
Runchu Tian, Yining Ye, Yujia Qin, Xin Cong, Yankai Lin, Yinxu Pan, Yesai Wu, Hui Haotian, Liu Weichuan, Zhiyuan Liu, Maosong Sun
| Challenge: | Large language models (LLMs) have demonstrated exceptional coding capabilities, but their debugging capabilities remain relatively unexplored. |
| Approach: | They propose a debugging benchmark consisting of 4,253 LLMs with four major bug categories and 18 minor types in C++, Java, and Python. |
| Outcome: | The proposed benchmark covers four major bug categories and 18 minor types in C++, Java, and Python. |
Distribution Calibration for Out-of-Domain Detection with Bayesian Approximation (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods for OOD detection are overconfident for OD samples . lack of labeled OOD examples leads to poor prior knowledge about these unknown intents, making it challenging to detect OOD samples. |
| Approach: | They propose a Bayesian OOD detection framework to calibrate distribution uncertainty using Monte-Carlo Dropout. |
| Outcome: | The proposed framework gains 33.33% OOD F1 improvements with increasing only 0.41% inference time compared to previous methods. |
Rewarding the Rare: Uniqueness-Aware RL for Creative Problem Solving in LLMs (2026.findings-acl)
Copied to clipboard
Zhiyuan Hu, Yucheng Wang, Yufei He, Jiaying Wu, Yilun Zhao, See-Kiong Ng, Cynthia Breazeal, Anh Tuan Luu, Hae Won Park, Bryan Hooi
| Challenge: | Reinforcement learning (RL) is a paradigm for post-training large language models, but it suffers from exploration collapse . a new study finds that RL fails to reward correct solutions that exhibit rare high-level strategies . |
| Approach: | They propose a method that rewards correct solutions that exhibit rare high-level strategies by clustering rollouts according to their high- level solution strategies. |
| Outcome: | The proposed approach improves pass@k across large sampling budgets and increases area under the pass@K curve (AUC@K) without sacrificing pass@1. |