HqeKV: Towards Hybrid Quantization and Eviction for KV Cache in Long-Context LLM Inference (2026.findings-acl)
Copied to clipboard
| Challenge: | autoregressive inference requires repeated computation across transformer layers. |
| Approach: | They propose a hybrid compression framework built on both quantization and eviction . they propose varying importance metric and flexible conversion policies to reduce memory overhead . |
| Outcome: | The proposed framework outperforms state-of-the-art methods under memory constraints. |
Similar Papers
TailorKV: A Hybrid Framework for Long-Context Inference via Tailored KV Cache Optimization (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing work mitigates memory overhead by offloading or compressing the Key-Value cache. |
| Approach: | They propose a method that integrates quantization and offloading into a generative large language model by using a hybrid compression method. |
| Outcome: | The proposed method outperforms the state-of-the-art in long-context evaluations. |
OjaKV: Context-Aware Online Low-Rank KV Cache Compression (2026.findings-acl)
Copied to clipboard
Yuxuan Zhu, David H. Yang, Mohammad Mohammadi Amiri, Keerthiram Murugesan, Tejaswini Pedapati, Pin-Yu Chen
| Challenge: | Existing methods for inference use static, offline-learned subspaces that perform poorly under distribution shifts. |
| Approach: | They propose a framework that integrates a storage policy with an online subspace adaptation to preserve key-value tokens in full rank as high-fidelity anchors. |
| Outcome: | Experiments show that OjaKV maintains or improves zero-shot accuracy at high compression ratios, achieving the strongest gains on long-context benchmarks requiring complex reasoning. |
KV Cache Compression, But What Must We Give in Return? A Comprehensive Benchmark of Long Context Capable Approaches (2024.findings-emnlp)
Copied to clipboard
Jiayi Yuan, Hongyi Liu, Shaochen Zhong, Yu-Neng Chuang, Songchen Li, Guanchu Wang, Duy Le, Hongye Jin, Vipin Chaudhary, Zhaozhuo Xu, Zirui Liu, Xia Hu
| Challenge: | Long context capability is a crucial competency for large language models as it mitigates the human struggle to digest long-form texts. |
| Approach: | They propose to evaluate 10+ state-of-the-art approaches for long context-capable LLMs. |
| Outcome: | The proposed methods are compared against 10+ state-of-the-art approaches across seven categories of long context tasks. |
MiniKV: Pushing the Limits of 2-Bit KV Cache via Compression and System Co-Design for Efficient Long Context Inference (2025.findings-acl)
Copied to clipboard
| Challenge: | State-of-the-art 2-bit KV cache quantization methods achieve excellent results in accelerating LLM inference while retaining accuracy on long context tasks. |
| Approach: | They propose a method based on 2-bit KV cache quantization with adaptive KV policies that retain LLM accuracy with only a subset of KV states. |
| Outcome: | The proposed method outperforms state-of-the-art methods on a wide range of long context tasks while retaining accuracy. |
HybridKV: Hybrid KV Cache Compression for Efficient Multimodal Large Language Model Inference (2026.acl-long)
Copied to clipboard
| Challenge: | Multimodal Large Language Models (MLLMs) are hindered by the rapid growth of key–value (KV) caches. |
| Approach: | They propose a hybrid KV cache compression framework that reduces KV memory by 7.9 and speeds up decoding by 1.52. |
| Outcome: | Experiments on 11 multimodal benchmarks show that HYBRIDKV cuts KV cache memory by 7.9 and speeds up decoding by 1.52. |
ContrastKV: Robust KV Cache Eviction via Contrastive Signal Fusion for Multi-Query Generalization (2026.acl-long)
Copied to clipboard
| Challenge: | Existing query-agnostic approaches rely on a single proxy query, leading to fragile eviction decisions under high evict ratios. |
| Approach: | They propose a query-agnostic KV cache eviction algorithm that exploits complementary semantic and non-semantic signals. |
| Outcome: | Experiments show that the proposed algorithm outperforms state-of-the-art methods while retaining up to 92% accuracy with only 20% of the KV cache budget. |
GraphKV: Breaking the Static Selection Paradigm with Graph-Based KV Cache Eviction (2025.emnlp-main)
Copied to clipboard
| Challenge: | Efficient Key-Value (KV) cache management is essential for processing long text sequences in large language models (LLMs). |
| Approach: | They propose a graph-based framework that redefines token selection for KV cache compression. |
| Outcome: | The proposed framework can be used in existing KV cache eviction methods such as SnapKV and PyramidKV in a plug-and-play manner. |
VecInfer: Efficient LLM Inference with Low-Bit KV Cache via Outlier-Suppressed Vector Quantization (2026.acl-long)
Copied to clipboard
| Challenge: | Existing quantization methods for large language models suffer performance degradation at ultra-low bit-widths due to key cache outliers. |
| Approach: | They propose a vector quantization method that suppresses outliers in the key cache and reduces memory access overhead. |
| Outcome: | The proposed method outperforms baseline quantization methods across long-context understanding and mathematical reasoning tasks while minimizing memory access overhead. |
MadaKV: Adaptive Modality-Perception KV Cache Eviction for Efficient Multimodal Long-Context Inference (2025.acl-long)
Copied to clipboard
Kunxi Li, Zhonghua Jiang, Zhouzhou Shen, ZhaodeWang ZhaodeWang, Chengfei Lv, Shengyu Zhang, Fan Wu, Fei Wu
| Challenge: | Existing KV cache eviction methods fail to capture modality-specific information, resulting in suboptimal performance. |
| Approach: | They propose a modality-adaptive key-value (KV) cache eviction strategy to enhance the efficiency of multimodal large language models in long-context inference. |
| Outcome: | The proposed method reduces the KV cache memory footprint and model inference latency while maintaining high accuracy across multimodal long-context tasks. |
TaDA: Training-free recipe for Decoding with Adaptive KV Cache Compression and Mean-centering (2025.acl-industry)
Copied to clipboard
| Challenge: | key-value caches in large language models consume memory, posing a major challenge for scalable deployment. |
| Approach: | They propose a training-free recipe for KV cache compression with quantization precision that adapts to error sensitivity across layers and a mean centering to eliminate separate outlier handling. |
| Outcome: | The proposed technique reduces the KV cache memory footprint to 27% of the original 16-bit baseline while achieving comparable accuracy. |