Papers by Junwan Kim
Reducing Peak Memory Usage for Modern Multimodal Large Language Model Pipelines (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to reduce memory usage of multimodal large language models rely on storing large numbers of vision tokens in the key–value cache . however, such compression is typically only applied after all inputs are processed, resulting in high peak memory usage during the prefill stage. |
| Approach: | They propose a sequential input-compression mechanism that enforces a fixed memory budget by performing structure-aware key–value cache compression during the prefill stage. |
| Outcome: | The proposed approach reduces peak memory usage while maintaining generative performance with only minimal degradation, enabling more practical and memory-efficient multimodal inference. |