Papers by Alok Prakash
TETRIS: Optimal Draft Token Selection for Batch Speculative Decoding (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods that optimize for a single request or a group of requests as a whole only select the most promising draft tokens to be accepted when verified in parallel. |
| Approach: | They propose a method that optimizes the total throughput of batch speculative decoding in multi-request settings by actively selecting the most promising draft tokens to be accepted when verified in parallel. |
| Outcome: | The proposed method outperforms baseline speculative decoding and existing methods that dynamically select draft tokens, leading to a more efficient batch inference in large language models. |