Papers by Zhiheng Liu
Soundwave: Less is More for Speech-Text Alignment in LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Existing end-to-end speech large language models rely on large-scale annotated data for training, while data-efficient training has not been discussed in depth. |
| Approach: | They propose a training strategy and a novel architecture to address representation space gap and sequence length inconsistency in speech and text. |
| Outcome: | The proposed model outperforms other advanced speech LLMs in speech translation and AIR-Bench speech tasks with only a fraction of the training data. |
AgentGym2: Benchmarking Large Language Model Agents in De-Idealized Real-World Environments (2026.acl-long)
Copied to clipboard
Zhiheng Xi, Dingwen Yang, Jiaqi Liu, Jixuan Huang, Honglin Guo, Baodai Huang, Tinggang Chen, Qi Zhang, Zhonghang Lu, Chenyu Liu, Jiajun Sun, Jiazheng Zhang, Dingwei Zhu, Xin Guo, Junzhe Wang, Zhihao Zhang, Yuming Yang, Junjie Ye, Minghe Gao, Dongrui Liu, Jiaming Ji, Guohao Li, Tao Gui, Qi Zhang, Xuanjing Huang
| Challenge: | Existing benchmarks evaluate agents in simplified, idealized settings, relying on pre-packaged tool interfaces, overlooking critical steps, and assume inputs are clean and fully specified. |
| Approach: | They propose a framework that evaluates language agents in simplified, idealized settings . they show that even SOTA systems like Gemini and GPT-5 struggle on AgentGym2 . |
| Outcome: | Experiments on 15 proprietary and open-source models show that even SOTA systems like Gemini and GPT-5 struggle on AgentGym2 . |
LoRAMoE: Alleviating World Knowledge Forgetting in Large Language Models via MoE-Style Plugin (2024.acl-long)
Copied to clipboard
Shihan Dou, Enyu Zhou, Yan Liu, Songyang Gao, Wei Shen, Limao Xiong, Yuhao Zhou, Xiao Wang, Zhiheng Xi, Xiaoran Fan, Shiliang Pu, Jiang Zhu, Rui Zheng, Tao Gui, Qi Zhang, Xuanjing Huang
| Challenge: | Experimental results show that, as the instruction data increases, LoRAMoE can significantly improve the ability to process downstream tasks, while maintaining the world knowledge stored in the LLM. |
| Approach: | They propose a framework that introduces several low-rank adapters and integrates them by using a router network to freeze the backbone model and force a portion of LoRAs to focus on leveraging world knowledge to solve downstream tasks. |
| Outcome: | The proposed framework freezes the backbone model and forces a portion of LoRAs to focus on leveraging world knowledge to solve downstream tasks, to alleviate world knowledge forgetting. |
Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Models (2026.findings-acl)
Copied to clipboard
Hengyuan Zhang, Zhihao Zhang, Ercong Nie, Mingyang Wang, Zunhai Su, Yiwei Wang, Qianli Wang, Shuzhou Yuan, Xufeng Duan, Qibo Xue, Zeping Yu, Chenming Shang, Xiao Liang, Jing Xiong, Hui Shen, Chaofan Tao, Zhengwu Liu, Senjie Jin, Zhiheng Xi, Dongdong Zhang, Sophia Ananiadou, Tao Gui, Ruobing Xie, Hayden Kwok-Hay So, Hinrich Schuetze, Xuanjing Huang, Qi Zhang, Ngai Wong
| Challenge: | Existing literature on mechanistic interpretation (MI) treats it as an observational science, leaving practical applications underexplored. |
| Approach: | They propose a survey structured around the pipeline to identify and improve MI models. |
| Outcome: | The proposed framework enables tangible improvements in Alignment, Capability, and Efficiency. |
Hybrid Hierarchical Retrieval for Open-Domain Question Answering (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent work shows that dense hierarchical retrieval (DHR) can outperform dense passage retrieval. |
| Approach: | They propose a framework that applies sparse, dense and a combination of them to document and passage retrieval. |
| Outcome: | The proposed framework can outperform dense hierarchical retrieval (DHR) and sparse retrievers (BM25) on open-domain question answering (ODQA) datasets with an average improvement of 4.69% on recall@100 over DHR. |
AgentV-RL: Scaling Reward Modeling with Agentic Verifier (2026.findings-acl)
Copied to clipboard
Jiazheng Zhang, Ziche Fu, Zhiheng Xi, Wenqing Jing, Mingxu Chai, Wei He, Guoqiang Zhang, Chenghao Fan, Chenxin An, Wenxiang Chen, Zhicheng Liu, Haojie Pan, Dingwei Zhu, Tao Gui, Qi Zhang, Xuanjing Huang
| Challenge: | Existing approaches to improve LLM reasoning are limited in complex domains and lack external grounding makes verifiers unreliable on computation-intensive tasks. |
| Approach: | They propose a framework that transforms reward modeling into a multi-turn, tool-augmented deliberative process. |
| Outcome: | The proposed framework surpasses state-of-the-art ORMs by 25.2% under parallel and sequential TTS. |
Self-Polish: Enhance Reasoning in Large Language Models via Problem Refinement (2023.findings-emnlp)
Copied to clipboard
Zhiheng Xi, Senjie Jin, Yuhao Zhou, Rui Zheng, Songyang Gao, Jia Liu, Tao Gui, Qi Zhang, Xuanjing Huang
| Challenge: | Existing prompting methods have been used to enhance multistep reasoning capabilities of large language models, but they have overlooked the potential of formulating higher-quality problems. |
| Approach: | They propose a method that starts from the problem side and refines problems to be more comprehensible and solvable for models. |
| Outcome: | The proposed method achieves notable and consistent effectiveness on five reasoning benchmarks across different models. |
Multi-Programming Language Sandbox for LLMs (2025.acl-demo)
Copied to clipboard
Shihan Dou, Jiazheng Zhang, Jianxiang Zang, Yunbo Tao, Weikang Zhou, Haoxiang Jia, Shichun Liu, Yuming Yang, Shenxi Wu, Zhiheng Xi, Muling Wu, Rui Zheng, Changze Lv, Limao Xiong, Shaoqing Zhang, Lin Zhang, Wenyu Zhan, Rongxiang Weng, Jingang Wang, Xunliang Cai, Yueming Wu, Ming Wen, Yixin Cao, Tao Gui, Xipeng Qiu, Qi Zhang, Xuanjing Huang
| Challenge: | MPLSandbox is an out-of-the-box multi-programming language sandbox designed to provide unified and comprehensive feedback from compiler and analysis tools for Large Language Models (LLMs). |
| Approach: | They propose a multi-programming language sandbox that provides unified feedback from compilers and analysis tools for Large Language Models. |
| Outcome: | The proposed multi-language sandbox can provide comprehensive feedback from compilers and analysis tools for large language models (LLMs). |
Hetero-Designer: Automated Design of Multi-Agent Systems with Heterogeneous LLMs (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to design LLM-based Multi-agent systems are constrained by homogeneous LLMs. |
| Approach: | They propose an automated design of heterogeneous-LLMs-based MAS with a binary-star transformer and an autoregressive graph generation pipeline. |
| Outcome: | The proposed pipeline is high-performing on various benchmarks and extensible to unseen LLMs and roles. |
VENUS: A VLLM-driven Video Content Discovery System for Real Application Scenarios (2025.emnlp-industry)
Copied to clipboard
Minyi Zhao, Yi Liu, Jianfeng Wen, Boshen Zhang, Hailang Chang, Zhiheng Ouyang, Jie Wang, Wensong He, Shuigeng Zhou
| Challenge: | Video Content Discovery (VCD) is to identify specific videos defined by a pre-specified text policy. |
| Approach: | They propose a Vision-Language Large Model-driven video content discovery system called VENUS to solve these problems. |
| Outcome: | The proposed system generates high-quality, VCD-specific data for model training and extends it to support it better. |
LLMEval-Fair: A Large-Scale Longitudinal Study on Robust and Fair Evaluation of Large Language Models (2026.acl-long)
Copied to clipboard
Ming Zhang, Yujiong Shen, Jingyi Deng, Yuhui Wang, Huayu Sha, Kexin Tan, Qiyuan Peng, Yue Zhang, Junzhe Wang, Shichun Liu, Yueyuan Huang, Jingqi Tong, Changhao Jiang, Yilong Wu, Zhihao Zhang, Mingqi Wu, Mingxu Chai, Zhiheng Xi, Shihan Dou, Tao Gui, Qi Zhang, Xuanjing Huang
| Challenge: | Existing evaluation of Large Language Models on static benchmarks is vulnerable to data contamination and leaderboard overfitting. |
| Approach: | LLMEval-Fair framework provides a framework for dynamic evaluation of Large Language Models . evaluators use a proprietary bank of 220k graduate-level questions to analyze model data . |
| Outcome: | LLMEval-Fair provides robust and credible evaluation framework for Large Language Models . it provides a strong empirical validation for the dynamic evaluation paradigm . |
A Systematic Assessment of Language Models with Linguistic Minimal Pairs in Chinese (2026.tacl-1)
Copied to clipboard
Yikang Liu, Yeting Shen, Hongao Zhu, Lilong Xu, Zhiheng Qian, Siyuan Song, Kejia Zhang, Jialong Tang, Pei Zhang, Baosong Yang, Rui Wang, Hai Hu
| Challenge: | Using sub-linear length normalized log-probabilities (SLLN-LP), we find unequal lengths of sentences in minimal pairs difficult for LMs even up to 32B parameters. |
| Approach: | They propose to use ZhoBLiMP as a linguistic minimal pair benchmark for Chinese language models to mitigate biases. |
| Outcome: | The proposed metric mitigates biases in Chinese language models with over 100 paradigms . Anaphor, Quantifiers, and Ellipsis are difficult for LMs even up to 32B parameters . |
RobustQA: Benchmarking the Robustness of Domain Adaptation for Open-Domain Question Answering (2023.findings-acl)
Copied to clipboard
Rujun Han, Peng Qi, Yuhao Zhang, Lan Liu, Juliette Burger, William Yang Wang, Zhiheng Huang, Bing Xiang, Dan Roth
| Challenge: | Existing ODQA datasets consist mainly of Wikipedia corpus, and are insufficient to study models’ generalizability across diverse domains. |
| Approach: | They propose a benchmark to evaluate ODQA's domain robustness using Wikipedia corpus . they annotate QA pairs in retrieval datasets with rigorous quality control . |
| Outcome: | The proposed benchmark improves model performance on annotated QA pairs in retrieval datasets with rigorous quality control. |
Characterizing the Impacts of Instances on Robustness (2023.findings-acl)
Copied to clipboard
Rui Zheng, Zhiheng Xi, Qin Liu, Wenbin Lai, Tao Gui, Qi Zhang, Xuanjing Huang, Jin Ma, Ying Shan, Weifeng Ge
| Challenge: | Existing defense approaches focus on developing new model structures or training algorithms, but they do little to tap the potential of training instances. |
| Approach: | They propose a method that can distinguish between robust and non-robust instances according to the model’s sensitivity to perturbations on individual instances during training. |
| Outcome: | The proposed method can distinguish between robust and non-robust instances according to the model’s sensitivity to perturbations on individual instances during training. |
Improving Cross-task Generalization of Unified Table-to-text Models with Compositional Task Configurations (2023.findings-acl)
Copied to clipboard
Jifan Chen, Yuhao Zhang, Lan Liu, Rui Dong, Xinchi Chen, Patrick Ng, William Yang Wang, Zhiheng Huang
| Challenge: | Existing methods for multitask learning typically use a dataset name as input prefix, which limits the effectiveness of multitask training. |
| Approach: | They propose compositional task configurations, a set of prompts prepended to the encoder to improve cross-task generalization of unified models. |
| Outcome: | The proposed model outperforms the UnifiedSKG baseline by noticeable margins in both in-domain and zero-shot settings. |
Toward Optimal LLM Alignments Using Two-Player Games (2025.findings-emnlp)
Copied to clipboard
Rui Zheng, Hongyi Guo, Zhihan Liu, Xiaoying Zhang, Yuanshun Yao, Xiaojun Xu, Zhaoran Wang, Zhiheng Xi, Tao Gui, Qi Zhang, Xuanjing Huang, Yang Liu, Hang Li
| Challenge: | Alignment of large language models (LLM) is a process that ensures the model’s responses to user prompts align with human intentions and social values. |
| Approach: | They propose an alignment method based on a two-agent game consisting of an adversarial agent and a defensive agent. |
| Outcome: | The proposed method improves on a two-agent game with an adversarial agent and a defensive agent. |
MM-Doc-R1: Training Agents for Long Document Visual Question Answering through Multi-turn Reinforcement Learning (2026.findings-acl)
Copied to clipboard
Jiahang Lin, Kai Hu, Binghai Wang, Yuhao Zhou, Zhiheng Xi, Honglin Guo, Shichun Liu, Junzhe Wang, Shihan Dou, Enyu Zhou, Hang Yan, Zhenhua Han, Tao Gui, Qi Zhang, Xuanjing Huang
| Challenge: | Existing work on long document visual question answering is based on Retrieval-Augmented Generation (RAG) where textual or visual content is encoded into embeddings and relevance is determined by similarity scores with respect to the original query. |
| Approach: | They propose a framework that employs an agentic, vision-aware workflow to address long document visual question answering through iterative information discovery and synthesis. |
| Outcome: | The proposed framework outperforms existing RL systems by 10.4% on the MMLongbench-Doc benchmark and demonstrates superior training performance over GRPO. |
LLMEval-Med: A Real-world Clinical Benchmark for Medical LLMs with Physician Validation (2025.findings-emnlp)
Copied to clipboard
Ming Zhang, Yujiong Shen, Zelin Li, Huayu Sha, Binze Hu, Yuhui Wang, Chenhao Huang, Shichun Liu, Jingqi Tong, Changhao Jiang, Mingxu Chai, Zhiheng Xi, Shihan Dou, Tao Gui, Qi Zhang, Xuanjing Huang
| Challenge: | Current medical benchmarks have limitations in question design, data sources and evaluation methods. |
| Approach: | They propose a new benchmark covering five core medical areas . it includes 2,996 questions created from real-world electronic health records . |
| Outcome: | The proposed model covers five core medical areas and includes 2,996 questions created from real-world electronic health records and expert-designed clinical scenarios. |
Language Agnostic Multilingual Information Retrieval with Contrastive Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | Annotated training data is costly to obtain in many languages . |
| Approach: | They propose a semantic contrastive loss to align parallel sentences that share the same semantics in different languages and a language contrastive gain to leverage parallel sentence pairs to remove language-specific information from non-parallel corpora. |
| Outcome: | The proposed model improves retrieval performance while requiring less computational effort. |
Self-Demos: Eliciting Out-of-Demonstration Generalizability in Large Language Models (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods that rely on limited demos and out-of-demonstration (OOD) queries fail when faced with out- of-demotion queries. |
| Approach: | They propose a query-aware prompting method that elicits the inherent generalizability of large language models by query-based demo generation. |
| Outcome: | The proposed method outperforms state-of-the-art methods in the OOD setting and two public math benchmarks. |
ABC-Bench: Benchmarking Agentic Backend Coding in Real-World Development (2026.findings-acl)
Copied to clipboard
Jie Yang, Honglin Guo, Li Ji, Jiazheng Zhou, Rui Zheng, Zhikai Lei, Shuo Zhang, Zhiheng Xi, Shichun Liu, Yuxin Wang, Bo Wang, Yining Zheng, Tao Gui, Xipeng Qiu
| Challenge: | Large Language Models (LLMs) have redefined the role of AI in software engineering . current benchmarks focus on localized code generation, but neglect dynamic, full-process requirements of real-world engineering. |
| Approach: | They propose a benchmark to evaluate agentic backend coding within a realistic, executable workflow. |
| Outcome: | The ABC-Bench benchmark evaluates agentic backend coding within a realistic, executable workflow. |
StepCoder: Improving Code Generation with Reinforcement Learning from Compiler Feedback (2024.acl-long)
Copied to clipboard
Shihan Dou, Yan Liu, Haoxiang Jia, Enyu Zhou, Limao Xiong, Junjie Shan, Caishuang Huang, Xiao Wang, Xiaoran Fan, Zhiheng Xi, Yuhao Zhou, Tao Ji, Rui Zheng, Qi Zhang, Tao Gui, Xuanjing Huang
| Challenge: | Existing work integrates reinforcement learning with compiler feedback to enhance code generation quality but the long code generated by LLMs makes RL exploration ineffective. |
| Approach: | They propose a framework that integrates reinforcement learning and compiler feedback to enhance code generation quality. |
| Outcome: | The proposed framework outperforms state-of-the-art approaches in corresponding benchmarks and integrates reinforcement learning with compiler feedback to improve code generation quality. |
Tokenization Consistency Matters for Generative Models on Extractive NLP Tasks (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Pretrained sequence-to-sequence (seq2sequ) models have been widely used to solve extractive tasks, where parts of the input are extracted to form the desired output. |
| Approach: | They propose a simple fix to tokenization inconsistency that damages extractive nature of generative models by causing performance drop and hallucination. |
| Outcome: | The proposed model performs better in both in-domain and out-of-domain datasets with a notable average of +1.7 F1 gain when a BART model is trained on SQuAD and evaluated on 8 QA datasets. |
Are LLMs Rational Investors? A Study on the Financial Bias in LLMs (2025.findings-acl)
Copied to clipboard
Yuhang Zhou, Yuchen Ni, Zhiheng Xi, Zhangyue Yin, Yu He, Gan Yunhui, Xiang Liu, Zhang Jian, Sen Liu, Xipeng Qiu, Yixin Cao, Guangnan Ye, Hongfeng Chai
| Challenge: | Existing studies on biases within specific domains, such as finance, remain limited. |
| Approach: | They propose a framework to detect, detect, analyze and mitigate financial biases in large language models. |
| Outcome: | The proposed framework reduces bias by 68% for the most biased model, according to key metrics. |
TextFlint: Unified Multilingual Robustness Evaluation Toolkit for Natural Language Processing (2021.acl-demo)
Copied to clipboard
Xiao Wang, Qin Liu, Tao Gui, Qi Zhang, Yicheng Zou, Xin Zhou, Jiacheng Ye, Yongxin Zhang, Rui Zheng, Zexiong Pang, Qinzhuo Wu, Zhengyan Li, Chong Zhang, Ruotian Ma, Zichu Fei, Ruijian Cai, Jun Zhao, Xingwu Hu, Zhiheng Yan, Yiding Tan, Yuan Hu, Qiyuan Bian, Zhihua Liu, Shan Qin, Bolin Zhu, Xiaoyu Xing, Jinlan Fu, Yue Zhang, Minlong Peng, Xiaoqing Zheng, Yaqian Zhou, Zhongyu Wei, Xipeng Qiu, Xuanjing Huang
| Challenge: | Existing approaches to textual robustness evaluation focus on slightly modifying the input data, which maintains the original meaning and results in a different prediction. |
| Approach: | They propose a multilingual robustness evaluation toolkit for NLP that integrates universal text transformations, task-specific transformations and adversarial attack. |
| Outcome: | The toolkit includes universal text transformation, task-specific transformation, adversarial attack, subpopulation, and their combinations to provide comprehensive robustness analyses. |