Papers by Guy Boudoukh
FastDraft: How to Train Your Draft (2025.findings-acl)
Copied to clipboard
| Challenge: | Speculative Decoding relies on the availability of efficient draft models, which are often lacking due to a stringent constraint of vocabulary compatibility. |
| Approach: | They propose a novel approach for pre-training and aligning a draft model to any large language model by incorporating efficient pre-train and fine-tuning over synthetic datasets generated by the target model. |
| Outcome: | The proposed model can be trained on a single server with 8 Intel Gaudi 2 accelerators in under 24 hours and achieves 3x acceptance rate, block efficiency and 2x memory bound speedup. |