Papers by Jiwon Song
FastKV: Decoupling of Context Reduction and KV Cache Compression for Prefill-Decoding Acceleration (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) excel at handling long-context sequences, but require substantial prefill computation and key-value (KV) cache. |
| Approach: | They propose a KV cache compression framework that decouples prefill computation from decoding KV budget. |
| Outcome: | The proposed framework reduces latency in prefill and decoding by leveraging the stabilization of token importance in later layers. |