Papers by Yingmin Qiu
GuiLoMo: Allocating Experts and Ranks for LoRA-MoE via Bilevel Optimization with GuidedSelection Vectors (2025.findings-emnlp)
Copied to clipboard
Xinrong Chen, Hengyuan Zhang, Yingmin Qiu, Xiao Liang, Ziyue Li, Guanyu Wang, Weiping Li, Tong Mo, Hayden Kwok-Hay So, Ngai Wong
| Challenge: | Low-Rank Adaptation (LoRA) methods are efficient for a large language model with reduced computational costs. |
| Approach: | They propose a layer-wise expert numbers and ranks allocation strategy with GuidedSelection Vectors. |
| Outcome: | The proposed method achieves superior or comparable performance to all baselines on three backbone models. |