Papers by Feihu Jin
Two-Stage Regularization-Based Structured Pruning for LLMs (2026.acl-long)
Copied to clipboard
Mingkuan Feng, Jinyang Wu, Siyuan Liu, Shuai Zhang, Hongjian Fang, Ruihan Jin, Feihu Che, Pengpeng Shao, Zhengqi Wen, Jianhua Tao
| Challenge: | Structural pruning is a promising solution for large language models . prior structured pruning methods remove unimportant parameters based on certain metrics . |
| Approach: | They propose a structural pruning method that iteratively learns the weights of transformer layers by adding their l1-norm to the loss function. |
| Outcome: | The proposed pruning method outperforms strong layer-wise pruning methods without requiring retraining. |
Parameter-efficient Tuning for Large Language Model without Calculating Its Gradients (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent parameter-efficient tuning methods can only save 30% of training memory . gradient computation and backpropagation are still necessary for these methods . |
| Approach: | They propose a parameter-efficient tuning method that can be used to fine-tune large language models without calculating gradients. |
| Outcome: | The proposed method saves 30% of training memory and improves performance on large language models. |
Hi-ZFO: Hierarchical Zeroth- and First-Order LLM Fine-Tuning via Importance-Guided Tensor Selection (2026.findings-acl)
Copied to clipboard
| Challenge: | generative tasks require a high degree of exploratory capacity, but zeroth-order methods suffer from slow convergence . generative task-specific methods tend to converge toward local minima, causing noise and inefficient estimation . |
| Approach: | They propose a framework that synergizes FO precision with exploratory capability of ZO estimation. |
| Outcome: | The proposed framework synergizes precision of FO gradients with exploratory capability of ZO estimation. |