Papers by Ganghao Liu
Specialization without Sparsity: Efficient and Expressive Split-Path Experts for LLM Fine-Tuning (2026.findings-acl)
Copied to clipboard
| Challenge: | Parameter-efficient fine-tuning (PEFT) is a low-cost alternative to full fine-timing due to the massive overhead. |
| Approach: | They propose a Mixture-of-Experts approach that enhances specialization while maintaining low resource overhead. |
| Outcome: | The proposed approach outperforms or matches state-of-the-art methods on GLUE, GSM8K, MBPP, and a text rewriting task from SmolTalk. |