Papers by In Gim
Cacheback: Speculative Decoding With Nothing But Cache (2025.emnlp-main)
Copied to clipboard
| Challenge: | a recent study shows that large language models are unable to model locality in language. |
| Approach: | They propose a training-free and model-agnostic speculative decoding method that exploits locality in language to accelerate Large Language Models. |
| Outcome: | The proposed method achieves state-of-the-art performance among comparable methods . it leverages only LRU cache tables of token n-grams to generate draft sequences . |