Papers by Fuwei Yang
Jakiro: Boosting Speculative Decoding via Decoupled MoE (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods to accelerate large language model inference have a fundamental limitation: candidates at the same tree layer share identical feature representations, constraining diversity and diminishing overall effectiveness. |
| Approach: | They propose a decoupled mixture of experts (MoE) into a draft model to generate diverse tokens from distinct feature spaces. |
| Outcome: | The proposed approach achieves significant speedups over strong baselines, with notable improvements in non-greedy scenarios where token diversity is crucial. |