Papers by Changze Lv
TripTailor: A Real-World Benchmark for Personalized Travel Planning (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing evaluation metrics for travel planning rely on unrealistic simulated data . fewer than 10% of the itineraries generated by the latest state-of-the-art LLMs achieve human-level performance. |
| Approach: | They propose a benchmark for personalized travel planning in real-world scenarios . they identify several critical challenges in travel planning including feasibility and rationality . |
| Outcome: | The proposed benchmarks show that fewer than 10% of the itineraries generated by the latest state-of-the-art LLMs achieve human-level performance. |
Enhancing Model Privacy in Federated Learning with Random Masking and Quantization (2025.findings-emnlp)
Copied to clipboard
Zhibo Xu, Zhu JianHao, Jingwen Xu, Changze Lv, Zhenghua Wang, Zisu Huang, Xiaohua Wang, Muling Wu, Qi Qian, Xiaoqing Zheng, Xuanjing Huang
| Challenge: | federated learning approaches are limited by the complexity of large language models and the need for specialized expertise to protect intellectual property. |
| Approach: | They propose a federated learning approach that leverages random masking to obscure a subnetwork of model parameters and applies quantization to the remaining parameters. |
| Outcome: | The proposed approach maintains strong model performance in federated learning settings and achieves enhanced protection of model parameters compared to baseline methods. |
Aligning Large Language Models with Human Preferences through Representation Engineering (2024.acl-long)
Copied to clipboard
Wenhao Liu, Xiaohua Wang, Muling Wu, Tianlong Li, Changze Lv, Zixuan Ling, Zhu JianHao, Cenyuan Zhang, Xiaoqing Zheng, Xuanjing Huang
| Challenge: | Existing methods for achieving this alignment involve employing reinforcement learning from human feedback (RLHF) Existing approaches involve using RLHF to fine-tune LLMs based on human labels . however, RLRF is susceptible to instability during fine- tuning and presents challenges in implementation. |
| Approach: | They propose to use reinforcement learning from human feedback to fine-tune large language models with human preferences to achieve precise control of model behavior. |
| Outcome: | Experiments show that RAHF can be used to capture and manipulate representations to align with a broad spectrum of human preferences or values rather than being confined to a single concept or function. |
Parameter Efficient Multi-task Fine-tuning by Learning to Transfer Token-wise Prompts (2023.findings-emnlp)
Copied to clipboard
Muling Wu, Wenhao Liu, Jianhan Xu, Changze Lv, Zixuan Ling, Tianlong Li, Longtao Huang, Xiaoqing Zheng, Xuanjing Huang
| Challenge: | Prompt tuning has been proven to be successful on various tasks by incorporating a small number of trainable parameters while freezing large pre-trained language models. |
| Approach: | They propose a token-wise prompt tuning method that uses a bank of finer-grained soft prompt tokens to generate an instance-dependent prompt. |
| Outcome: | The proposed method performs far better than full parameter fine-tuned models and achieves state-of-the-art by tuning only 0.035% parameters on 14 datasets. |
Multi-Programming Language Sandbox for LLMs (2025.acl-demo)
Copied to clipboard
Shihan Dou, Jiazheng Zhang, Jianxiang Zang, Yunbo Tao, Weikang Zhou, Haoxiang Jia, Shichun Liu, Yuming Yang, Shenxi Wu, Zhiheng Xi, Muling Wu, Rui Zheng, Changze Lv, Limao Xiong, Shaoqing Zhang, Lin Zhang, Wenyu Zhan, Rongxiang Weng, Jingang Wang, Xunliang Cai, Yueming Wu, Ming Wen, Yixin Cao, Tao Gui, Xipeng Qiu, Qi Zhang, Xuanjing Huang
| Challenge: | MPLSandbox is an out-of-the-box multi-programming language sandbox designed to provide unified and comprehensive feedback from compiler and analysis tools for Large Language Models (LLMs). |
| Approach: | They propose a multi-programming language sandbox that provides unified feedback from compilers and analysis tools for Large Language Models. |
| Outcome: | The proposed multi-language sandbox can provide comprehensive feedback from compilers and analysis tools for large language models (LLMs). |
Tell Me What You Don’t Know: Enhancing Refusal Capabilities of Role-Playing Agents via Representation Space Analysis and Editing (2025.findings-acl)
Copied to clipboard
Wenhao Liu, Siyu An, Junru Lu, Muling Wu, Tianlong Li, Xiaohua Wang, Changze Lv, Xiaoqing Zheng, Di Yin, Xing Sun, Xuanjing Huang
| Challenge: | Role-playing Agents (RPAs) struggle to recognize and respond to hard queries that conflict with their role-play knowledge. |
| Approach: | They propose a lightweight representation editing approach that conveniently shifts conflicting requests to the rejection region, thereby enhancing the model’s refusal accuracy. |
| Outcome: | The proposed model improves RPAs’ refusal ability of conflicting requests while maintaining their general role-playing capabilities. |
Searching for Best Practices in Retrieval-Augmented Generation (2024.emnlp-main)
Copied to clipboard
Xiaohua Wang, Zhenghua Wang, Xuan Gao, Feiran Zhang, Yixin Wu, Zhibo Xu, Tianyuan Shi, Zhengyuan Wang, Shizheng Li, Qi Qian, Ruicheng Yin, Changze Lv, Xiaoqing Zheng, Xuanjing Huang
| Challenge: | Retrieval-augmented generation (RAG) techniques have proven to be effective in integrating up-to-date information, mitigating hallucinations, and enhancing response quality, especially in specialized domains. |
| Approach: | They propose several strategies for deploying RAG that balance performance and efficiency. |
| Outcome: | The proposed approaches can significantly enhance question-answering capabilities and accelerate the generation of multimodal content using a “retrieval as generation” strategy. |
UPLex: Fine-Grained Personality Control in Large Language Models via Unsupervised Lexical Modulation (2025.findings-emnlp)
Copied to clipboard
Tianlong Li, Wenhao Liu, Muling Wu, Shihan Dou, Zhenghua Wang, Changze Lv, Xiaohua Wang, Xiaoqing Zheng, Xuanjing Huang
| Challenge: | Personality is a crucial factor that shapes human communication patterns, thereby regulating the personalities of large language models (LLMs). |
| Approach: | They propose a method that uses an Unsupervisedly-Built Personalized Lexicon (UPL) during the decoding phase to manipulate LLM’s personality traits. |
| Outcome: | The proposed method can modulate the personality expression of large language models by dynamically altering their predicted probability of upcoming words in a pluggable fashion. |
Advancing Parameter Efficiency in Fine-tuning via Representation Editing (2024.acl-long)
Copied to clipboard
Muling Wu, Wenhao Liu, Xiaohua Wang, Tianlong Li, Changze Lv, Zixuan Ling, Zhu JianHao, Cenyuan Zhang, Xiaoqing Zheng, Xuanjing Huang
| Challenge: | Parameter Efficient Fine-Tuning (PEFT) has gained significant attention for its ability to achieve competitive results while updating only a small subset of trainable parameters. |
| Approach: | They propose a new approach to fine-tuning neural models that scales and biases the representation produced at each layer. |
| Outcome: | The proposed approach reduces the number of trainable parameters by a factor of 25,700 compared to full parameter fine-tuning and by . 32 compared with LoRA. |
Beyond Single Labels: Improving Conversational Recommendation through LLM-Powered Data Augmentation (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for enhancing recommendation quality face false negatives . only one "silly cop movie" is labeled as positive, leading to suboptimal recommendations . |
| Approach: | They propose a data augmentation framework that leverages an LLM-based semantic retriever to identify diverse and semantically relevant items and filter them by a relevance scorer to remove noisy candidates. |
| Outcome: | The proposed approach improves performance on two benchmark datasets and user simulators. |
VIB-Probe: Detecting and Mitigating Hallucinations in Vision-Language Models via Variational Information Bottleneck (2026.acl-long)
Copied to clipboard
| Challenge: | Existing hallucination detection methods rely on external verification tools . however, entanglement of visual-linguistic syntax and noise makes it difficult to detect hallucis . |
| Approach: | They propose a hallucination detection framework that leverages the Variational Information Bottleneck theory to detect hallucinic heads and to infer hallucication mitigation strategies. |
| Outcome: | The proposed framework outperforms baselines in hallucinations and noise detection environments. |
Mitigating Position Bias in Transformers via Layer-Specific Positional Embedding Scaling (2026.findings-acl)
Copied to clipboard
Zhenghua Wang, Yiran Ding, Changze Lv, Yixin Wu, Tianlong Li, Zhibo Xu, Muling Wu, Tianyuan Shi, Shizheng Li, Qi Qian, Xuanjing Huang, Xiaoqing Zheng
| Challenge: | Existing methods to address the "lost-in-the-middle" problem suffer from high latency or suboptimal hand-crafted scaling strategies. |
| Approach: | They propose a layer-specific positional embedding scaling method that assigns distinct scaling factors to each layer. |
| Outcome: | Experiments show that the proposed method mitigates positional attention bias and delivers consistent improvements across multiple long-context benchmarks. |
Revisiting Jailbreaking for Large Language Models: A Representation Engineering Perspective (2025.coling-main)
Copied to clipboard
Tianlong Li, Zhenghua Wang, Wenhao Liu, Muling Wu, Shihan Dou, Changze Lv, Xiaohua Wang, Xiaoqing Zheng, Xuanjing Huang
| Challenge: | Recent surge in jailbreaking attacks has revealed significant vulnerabilities in Large Language Models (LLMs) however, limited research into the underlying mechanisms that make LLMs vulnerable to such attacks has been conducted. |
| Approach: | They propose that LLMs' self-safeguarding capability is linked to specific activity patterns within their representation space. |
| Outcome: | The proposed models can be detected with a few pairs of contrastive queries, and the robustness can be manipulated by weakening or strengthening these patterns. |
Mitigating Hallucinations in VLMs: Enhancing Visual Attention via Head-Wise Perturbation (2026.findings-acl)
Copied to clipboard
| Challenge: | Vision–Language Models (VLMs) have demonstrated strong capabilities in tasks that require joint understanding of text and images. |
| Approach: | They propose a strategy that incorporates head-wise attention perturbation via continuous multiplicative noise coupled with a visual-guided loss focused on vision-sensitive text tokens to promote a more balanced attention distribution. |
| Outcome: | The proposed approach outperforms baseline models on three benchmarks and consistently outperformed the baseline model. |
Improving Continual Pre-training Through Seamless Data Packing (2025.findings-acl)
Copied to clipboard
| Challenge: | Empirical evaluations across various model architectures and corpus domains demonstrate the effectiveness of our method, outperforming baselines in 99% of all settings. |
| Approach: | They propose a method that uses a sliding window technique to pack data before continual pre-training to preserve contextual information and enhance model performance. |
| Outcome: | Empirical evaluations across various model architectures and corpus domains demonstrate the effectiveness of the proposed method outperforming baselines in 99% of settings. |
Promoting Data and Model Privacy in Federated Learning through Quantized LoRA (2024.findings-emnlp)
Copied to clipboard
Zhu JianHao, Changze Lv, Xiaohua Wang, Muling Wu, Wenhao Liu, Tianlong Li, Zixuan Ling, Cenyuan Zhang, Xiaoqing Zheng, Xuanjing Huang
| Challenge: | Existing federated learning frameworks require substantial data and computational resources to develop large language models. |
| Approach: | They propose a method that distributes a quantized version of the model’s parameters during training and combine it with a popular fine-tuning method to significantly reduce communication costs. |
| Outcome: | The proposed method enables accurate estimations for parameter updates while preventing clients from accessing a model whose performance is comparable to the centrally hosted one. |
Controllable Memory Usage: Balancing Anchoring and Innovation in Long-Term Human–Agent Interaction (2026.acl-long)
Copied to clipboard
Zisu Huang, Muzhao Tian, Xiaohua Wang, Jingwen Xu, Zhengkang Guo, Qi Qian, Kaitao Song, Jiakang Yuan, Changze Lv, Xiaoqing Zheng
| Challenge: | Existing systems that use memory as an "all-or-nothing" approach to memory usage are often static and rely on experience-following tendencies. |
| Approach: | They propose a framework that allows users to dynamically regulate memory reliance by adding context into the model's prompt. |
| Outcome: | The proposed model outperforms prompting and memory masking strategies in multiple scenarios. |