Papers by Shensian Syu
Hierarchical Speculative Decoding with Dynamic Window (2025.findings-naacl)
Copied to clipboard
| Challenge: | Speculative decoding (SD) uses an efficient draft model to generate multiple tokens . previous methods depend on simple heuristics to select K or dynamically adjust the window size . |
| Approach: | They propose a framework that allows a draft model to generate multiple tokens . they propose HSDDW, which allows the draft model autonomously decide when to stop generating tokens. |
| Outcome: | The proposed framework outperforms existing state-of-the-art methods on four datasets. |