Papers by Jianchao Yang
SwiftPrune: Hessian-Free Weight Pruning for Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a novel post-training pruning method relies on the Hessian matrix to perform pruning . current pruning methods are computationally intensive and lack performance due to second-order derivative calculations. |
| Approach: | They propose a Hessian-free weight pruning method that reduces computational burden . they use an Exponentially Weighted Moving Average technique to bypass weight sorting . |
| Outcome: | The proposed method achieves hardware-efficient model compression by eliminating computational intensive calculations. |
WISCA: A Lightweight Model Transition Method to Improve LLM Training via Weight Scaling (2026.findings-acl)
Copied to clipboard
Jiacheng Li, Jianchao Tan, Zhidong Yang, Pingwei Sun, Feiye Huo, Jiayu Qin, Xiangyu Zhang, Maoxin He, Guangming Tan, Weile Jia, Xunliang Cai, Tong Zhao
| Challenge: | Recent advances in training optimization for Transformer-based large language models lack systematic optimization of weight patterns during training. |
| Approach: | They propose a Weight Scaling method that rescales weights while preserving model outputs to improve model training efficiency and model quality. |
| Outcome: | The proposed method significantly improves convergence quality and loss reduction in LLMs with Grouped Query Attention architectures and LoRA fine-tuning tasks. |