Papers by Elliott Delaye
KV Pareto: Systems-Level Optimization of KV Cache and Model Compression for Long Context Inference (2026.eacl-industry)
Copied to clipboard
| Challenge: | Long-context Large Language Models (LLMs) face significant memory bottlenecks due to the linear growth of key-value (KV) cache with sequence length. |
| Approach: | They propose a framework that maps the trade-off frontier between total memory consumption and task accuracy across three complementary optimization techniques. |
| Outcome: | The proposed model-specific configurations achieve 68-78% total memory reduction with minimal (1-3%) accuracy degradation on long-context tasks. |