Papers by Yingtao Shen
River-LLM: Large Language Model Seamless Exit Based on KV Share (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods to reduce latency and speed up early exits are costly and impose significant cost and energy consumption. |
| Approach: | They propose a lightweight KV-Shared Exit River framework that allows the backbone’s missing KV cache to be naturally generated and preserved during the exit process. |
| Outcome: | The proposed framework achieves 1.71 to 2.16 speedup while maintaining high generation quality. |