Papers with StreamingLLM
EQUIP: EQUivariant preserving In-Place updates for Efficient Token Pruning (2026.acl-long)
Copied to clipboard
| Challenge: | Token-pruning methods cause "holes" in KV tensors, posing major challenges . equip reduces recomputation of rotation operations through in-place update, caching and re-indexing . |
| Approach: | They propose an EQUIP-based in-place token update mechanism that preserves the equivariance property of the operations performed in the attention computation. |
| Outcome: | EQUIP reduces recomputation of rotation operations and reduces eviction overheads . it achieves geomean speedups of 1.62 (or 1.47) over StreamingLLM and 3.45 ( or 1.86) |
The Pitfalls of KV Cache Compression (2026.acl-long)
Copied to clipboard
| Challenge: | Recent literature has shown minimal degradation of KV cache in multi-instruction prompts . authors show that certain instructions degrade much more rapidly with compression . |
| Approach: | They propose to change KV cache eviction policies to reduce the impact of KV evict bias . they propose to use a 'simple' evviction policy to reduce ejection bias if the LLM is a multi-instruction model . |
| Outcome: | The proposed methods show that certain instructions degrade much faster with compression, causing them to be ignored by the LLM. |