Papers with eviction
HqeKV: Towards Hybrid Quantization and Eviction for KV Cache in Long-Context LLM Inference (2026.findings-acl)
Copied to clipboard
| Challenge: | autoregressive inference requires repeated computation across transformer layers. |
| Approach: | They propose a hybrid compression framework built on both quantization and eviction . they propose varying importance metric and flexible conversion policies to reduce memory overhead . |
| Outcome: | The proposed framework outperforms state-of-the-art methods under memory constraints. |
REAL: REtrieval-reAsoning and Logic-constructed Attention Behaviors for Long-Context KV Cache Compression (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods to evict keyvalue caches ignore diverse behavior in failure cases, such as bias and distraction. |
| Approach: | They propose a method to analyze attention head behaviors in success and failure scenarios by maximizing signal-to-noise ratio and minimizing noise from bias and distraction. |
| Outcome: | The proposed method achieves comparable accuracy to the strongest baseline, HeadKV-R2 on LongBench v2 while requiring 32x less space. |