Papers by Zi Chen
CoBa: Convergence Balancer for Multitask Finetuning of Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing multi-task learning approaches for large language models fall short due to computational intensive or lack of simultaneous task convergence. |
| Approach: | They propose a new multi-task learning approach that dynamically adjusts task weights during the training process, ensuring that the validation loss of all tasks progresses towards convergence at an even pace. |
| Outcome: | The proposed approach improves the performance of large language models by up to 13% compared to the second-best approaches. |
Adaptive Feature-based Low-Rank Compression of Large Language Models via Bayesian Optimization (2024.findings-emnlp)
Copied to clipboard
Yixin Ji, Yang Xiang, Juntao Li, Qingrong Xia, Zi Ye, Xinyu Duan, Zhefeng Wang, Kehai Chen, Min Zhang
| Challenge: | Large language models require a balance between efficiency and performance. |
| Approach: | They propose a low-rank compression technique that reduces non-essential parameters by decomposing weight matrices into products of two low-ranked matrici. |
| Outcome: | The proposed method outperforms existing pruning and low-rank compression techniques in maintaining model performance at the same compression ratio. |
HiGoE: Hierarchical Graph of Evidence to Enhance Retrieval-Augmented Generation for Long-context Summarization (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for long-context summarization fail to capture high-level thematic structures and long-range dependencies. |
| Approach: | They propose a hierarchical Graph of Evidence to reduce hallucination and attention dilution by replacing unreliable chunk-based methods with a filtered proposition–evidence graph. |
| Outcome: | Experiments show that HiGoE surpasses baselines in quality and efficiency. |
Beyond Prompt Engineering: A Systematic Analysis of Prompt Lexical Sensitivity and Its Impacts on Quality (2026.findings-acl)
Copied to clipboard
Qipeng Xie, Zi Liang, Jiafei Wu, Yufei Chen, Weizheng Wang, Wenao Ma, Zhong Ming, Haiqin Yang, Kaishun Wu
| Challenge: | Existing studies on prompt engineering have focused on optimizing models for performance under stylistic perturbations. |
| Approach: | They conduct the first analysis of n-gram token-level mechanisms . they find that higher average performance is inherently associated with lower variance and greater stability. |
| Outcome: | The proposed model reduces the variance of the generated code by 40% . the proposed model is based on a large-scale dataset of 132,000 prompt variants . |
Making Every Step Effective: Jailbreaking Large Vision-Language Models Through Hierarchical KV Equalization (2025.findings-emnlp)
Copied to clipboard
| Challenge: | HKVE selectively accepts gradient optimization results based on the distribution of attention scores across different layers, ensuring that every optimization step positively contributes to the attack. |
| Approach: | They propose a framework that selectively accepts gradient optimization results based on the distribution of attention scores across different layers and selectively takes them into account when calculating the attack success rate. |
| Outcome: | The proposed framework outperforms existing methods by achieving success rates of 75.08% on MiniGPT4, 85.84% on LLaVA and 81.00% on Qwen-VL. |