When Efficiency Meets Safety: A Benchmark Security Analysis of KV Cache Compression in Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Key-Value (KV) caching is widely used in large language models to enable long-context inference efficiently, yet its security implications remain underexplored. |
| Approach: | They propose a history-aware, per-head feedback merging strategy that prevents safety degradation while maintaining efficiency. |
| Outcome: | The proposed strategy prevents safety degradation while maintaining efficiency. |
Similar Papers
KV Pareto: Systems-Level Optimization of KV Cache and Model Compression for Long Context Inference (2026.eacl-industry)
Copied to clipboard
| Challenge: | Long-context Large Language Models (LLMs) face significant memory bottlenecks due to the linear growth of key-value (KV) cache with sequence length. |
| Approach: | They propose a framework that maps the trade-off frontier between total memory consumption and task accuracy across three complementary optimization techniques. |
| Outcome: | The proposed model-specific configurations achieve 68-78% total memory reduction with minimal (1-3%) accuracy degradation on long-context tasks. |
Towards Efficient Large Language Model Serving: A Survey on System-Aware KV Cache Optimization (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are a powerful tool for high-performance inference serving. |
| Approach: | They focus on system-aware KV infrastructure for serving LLMs . they analyze cross-behavior co-design affinity and behavior-objective links . |
| Outcome: | The proposed key-value (KV) cache is crucial for low-latency, high-throughput LLM inference serving. |
KV Cache Compression, But What Must We Give in Return? A Comprehensive Benchmark of Long Context Capable Approaches (2024.findings-emnlp)
Copied to clipboard
Jiayi Yuan, Hongyi Liu, Shaochen Zhong, Yu-Neng Chuang, Songchen Li, Guanchu Wang, Duy Le, Hongye Jin, Vipin Chaudhary, Zhaozhuo Xu, Zirui Liu, Xia Hu
| Challenge: | Long context capability is a crucial competency for large language models as it mitigates the human struggle to digest long-form texts. |
| Approach: | They propose to evaluate 10+ state-of-the-art approaches for long context-capable LLMs. |
| Outcome: | The proposed methods are compared against 10+ state-of-the-art approaches across seven categories of long context tasks. |
SCOPE: Optimizing Key-Value Cache Compression in Long-context Generation (2025.acl-long)
Copied to clipboard
| Challenge: | Excessive compression during the prefill phase impairs comprehension of reasoning tasks . SCOPE is a framework that performs KV cache optimization during the decoding and prefill phases . |
| Approach: | They propose a framework that performs optimization during the prefill and decoding phases . they propose enabling a sliding strategy to select essential heavy hitters for the decoding phase . |
| Outcome: | Experiments show that SCOPE can optimize key-value cache for long-context generation tasks . the framework can preserve essential information while minimizing memory usage and transfer . |
ZigZagKV: Dynamic KV Cache Compression for Long-context Modeling based on Layer Uncertainty (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods to accelerate inference of Large Language models (LLMs) are limited in their ability to retain key tokens as input length increases. |
| Approach: | They propose a method that leverages layer uncertainty to allocate budget size for each layer to reduce memory usage. |
| Outcome: | The proposed method reduces memory usage of the KV caches to only 20% when compared to full KV inference while achieving nearly lossless performance. |
A Systematic Study of Cross-Layer KV Sharing for Efficient LLM Inference (2025.naacl-short)
Copied to clipboard
| Challenge: | Recent studies have shown that sharing key-value (KV) cache across layers is effective in efficient inference of large language models. |
| Approach: | They propose a unified framework that covers several recent methods and their novel variants to investigate cross-layer KV sharing. |
| Outcome: | The proposed framework achieves higher throughput and better performance when reducing the size of the key-value cache by 2 while maintaining competitive performance. |
REAL: REtrieval-reAsoning and Logic-constructed Attention Behaviors for Long-Context KV Cache Compression (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods to evict keyvalue caches ignore diverse behavior in failure cases, such as bias and distraction. |
| Approach: | They propose a method to analyze attention head behaviors in success and failure scenarios by maximizing signal-to-noise ratio and minimizing noise from bias and distraction. |
| Outcome: | The proposed method achieves comparable accuracy to the strongest baseline, HeadKV-R2 on LongBench v2 while requiring 32x less space. |
Anchoring the Cache: Mitigating Contextual Hallucination in KV-Compressed Long-Context Summarization (2026.acl-long)
Copied to clipboard
| Challenge: | Recent studies show that KV cache compression can increase hallucination scores in LLMs . modern LLM models support extremely long sequences, but their impact on model hallucinosity remains underexplored. |
| Approach: | They propose a decoding-phase strategy that selectively removes generated KV pairs from retrieval heads responsible for retrieving critical information from source context. |
| Outcome: | The proposed method reduces hallucination across multiple models and datasets while preserving computational efficiency. |
A Simple and Effective L_2 Norm-Based Strategy for KV Cache Compression (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to reduce the KV cache size involve fine-tuning the model to learn a compression strategy or leveraging attention scores to reduce sequence length. |
| Approach: | They find a correlation between the L2 norm and attention scores over cached KV pairs . they compress the KV cache based on the L1 norm of key embeddings . |
| Outcome: | The proposed approach reduces the KV cache size by 50% on language modelling and needle-in-a-haystack tasks and 90% on passkey retrieval tasks without losing accuracy. |
The Pitfalls of KV Cache Compression (2026.acl-long)
Copied to clipboard
| Challenge: | Recent literature has shown minimal degradation of KV cache in multi-instruction prompts . authors show that certain instructions degrade much more rapidly with compression . |
| Approach: | They propose to change KV cache eviction policies to reduce the impact of KV evict bias . they propose to use a 'simple' evviction policy to reduce ejection bias if the LLM is a multi-instruction model . |
| Outcome: | The proposed methods show that certain instructions degrade much faster with compression, causing them to be ignored by the LLM. |