Papers by Brian Park
A Drop-In Solution for On-the-Fly Adaptation of Speculative Decoding in Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are highly memory-intensive when performing real-time inference. |
| Approach: | They propose a technique that allows for speculative decoding to be run on the fly to maximize the efficiency of LLM inferences. |
| Outcome: | The proposed solution can lead to 3.55-16.48% speed improvement over the standard speculative decoding, and 1.2-3.4 over the default LLMs. |