Papers by Yunbo Hou
From Pseudo-Balancing to True Specialization: Memory-Aware Routing for Mixture-of-Experts (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to optimize expert-centered load balancing fail to account for pseudo-balance phenomenon . severe knowledge overlap among experts leads to redundant representations and inefficient parameter utilization . |
| Approach: | They propose a method that prioritizes expert utilization over semantic alignment . they use memory-aware routing to ensure expert load balancing is consistent . |
| Outcome: | Experimental results show that MAR improves expert specialization by 35% and accuracy by 2%-25% . MAR matches baseline performance with only half the experts . |