Papers by Ilja Baumann
Optimized Speculative Sampling for GPU Hardware Accelerators (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large foundational speech and language models require more memory and computational resources to generate long sequences. |
| Approach: | They propose to optimize speculative sampling for parallel hardware accelerators by combining multiple GPU threads to reduce profiling time. |
| Outcome: | The proposed approach improves profiling time from 6% to 13% without compromising accuracy. |