Papers by Zhe Cao
Completely Modular Fine-tuning for Dynamic Language Adaptation (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing studies on multilingual fine-tuning with a fixed set of languages lack dynamic adaptability to new languages. |
| Approach: | They propose a modular fine-tuning pipeline that enables dynamic language adaptation for LLMs by first training English-centric adapters for each language separately and then merging them for arbitrary-direction translation. |
| Outcome: | The proposed pipeline achieves 86% performance over traditional fine-tuning on four languages, while training only 0.1% parameters and relying on English as a bridge language without catastrophic forgetting. |
Privacy Risks of Intermediate Representations: Attribute Inference in Distributed LLM Inference (2026.findings-acl)
Copied to clipboard
| Challenge: | Distributed LLMs avoid raw inputs by transmitting intermediate hidden states, a practice widely assumed to preserve privacy. |
| Approach: | They propose a distributed inference framework that transmits intermediate hidden states to avoid sending raw inputs by exposing sensitive user attributes. |
| Outcome: | The proposed approach achieves Top-1 accuracy of 0.997 on CMS, 0.980 on Skytrax, and 0.986 on ECHR. |
CLUE: A Chinese Language Understanding Evaluation Benchmark (2020.coling-main)
Copied to clipboard
Liang Xu, Hai Hu, Xuanwei Zhang, Lu Li, Chenjie Cao, Yudong Li, Yechen Xu, Kai Sun, Dian Yu, Cong Yu, Yin Tian, Qianqian Dong, Weitang Liu, Bo Shi, Yiming Cui, Junyi Li, Jun Zeng, Rongzhao Wang, Weijian Xie, Yanting Li, Yina Patterson, Zuoyu Tian, Yiwen Zhang, He Zhou, Shaoweihua Liu, Zhe Zhao, Qipeng Zhao, Cong Yue, Xinrui Zhang, Zhengliang Yang, Kyle Richardson, Zhenzhong Lan
| Challenge: | Existing language evaluation benchmarks for English are limited to English . lack of such benchmarks makes it difficult to replicate success in other languages . |
| Approach: | They introduce a large-scale Chinese language understanding evaluation benchmark . the benchmark uses a set of current state-of-the-art pre-trained Chinese models . |
| Outcome: | The first large-scale Chinese Language Understanding Evaluation (CLUE) benchmark is released . the benchmark evaluates models across a wide range of tasks on original Chinese text . existing language evaluation benchmarks are mostly limited to English . |
Exploring Intrinsic Language-specific Subspaces in Fine-tuning Multilingual Neural Machine Translation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Multilingual neural machine translation models support fine-tuning hundreds of languages simultaneously. |
| Approach: | They propose to fine-tune a language in its intrinsic subspace with a tiny fraction of entire parameters. |
| Outcome: | The proposed methods outperform full-parameter fine-tuning up to 2.25 spBLEU scores and reduce trainable parameters to 0.4% for high and medium-resource languages and 1.6% for low-resourced ones. |
Meeseeks: A Feedback-Driven, Iterative Self-Correction Benchmark evaluating LLMs’ Instruction Following Capability (2026.findings-acl)
Copied to clipboard
Jiaming Wang, Yunke Zhao, Peng Ding, Jun Kuang, Yibin Shen, Zhe Tang, Yilin Jin, ZongYu Wang, Xiaoyu Li, Xuezhi Cao
| Challenge: | Existing models lack the ability to adhere to instructions, resulting in suboptimal performance. |
| Approach: | They propose an automated iterative instruction-following benchmark with integrated feedback mechanism. |
| Outcome: | The proposed benchmark identifies erroneous components in model responses and provides feedback accurately. |
Attention Basin: Why Contextual Position Matters in Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are sensitive to the contextual position of information in input. |
| Approach: | They introduce Attention-Driven Reranking (AttnRank) which estimates a model’s intrinsic positional attention preferences using a small calibration set and reorders retrieved documents or few-shot examples to align the most salient content with these high-attention positions. |
| Outcome: | Experiments on multi-hop QA and few-shot in-context learning tasks show that AttnRank achieves substantial improvements across 10 large language models of varying architectures and scales, without modifying model parameters or training procedures. |
SOP-Maze: Evaluating Large Language Models on Complicated Business Standard Operating Procedures (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are widely deployed as domain-specific agents, but evaluation of their capabilities in such contexts has not been fully explored. |
| Approach: | They propose a benchmark to evaluate LLMs' ability to follow instructions and make decisions in real-world scenarios. |
| Outcome: | The proposed benchmark is constructed from real-world business data and adapted into 23 complex SOP scenarios. |
iPET: An Interactive Emotional Companion Dialogue System with LLM-Powered Virtual Pet World Simulation (2025.acl-demo)
Copied to clipboard
Zheyong Xie, Shaosheng Cao, Zuozhu Liu, Zheyu Ye, Zihan Niu, Chonggang Lu, Tong Xu, Enhong Chen, Zhe Xu, Yao Hu, Wei Lu
| Challenge: | Existing approaches to role-playing emotional companion products lack sustained personalization and contextual adaptability, limiting their effectiveness in real-world settings. |
| Approach: | They propose a virtual pet agent that can enhance user engagement through rich, dynamic pet behaviors and interactions tailored to individual preferences. |
| Outcome: | The proposed system has been deployed in a real-world, non-commercial product for 200 days and has demonstrated its effectiveness in practical applications. |