Papers by Zheda Mai
MEDA: Dynamic KV Cache Allocation for Efficient Multimodal Long-Context Inference (2025.naacl-long)
Copied to clipboard
| Challenge: | Long-context Multimodal Large Language Models (MLLMs) require substantial computational resources as their multimodal Key-Value (KV) cache grows with increasing input lengths, challenging memory and time efficiency. |
| Approach: | They propose a dynamic multimodal KV cache allocation strategy that dynamically allocating KV size based on attention entropy to better adapt to multimodal interactions. |
| Outcome: | The proposed model achieves up to 72% KV cache memory reduction and 2.82 faster decoding speeds while maintaining or enhancing performance on various multimodal tasks in a long context. |