Papers by Jiantong Jiang
Towards Efficient Large Language Model Serving: A Survey on System-Aware KV Cache Optimization (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are a powerful tool for high-performance inference serving. |
| Approach: | They focus on system-aware KV infrastructure for serving LLMs . they analyze cross-behavior co-design affinity and behavior-objective links . |
| Outcome: | The proposed key-value (KV) cache is crucial for low-latency, high-throughput LLM inference serving. |