Papers by Chao-Ting Chen
Efficient Beam Search for Large Language Models Using Trie-Based Decoding (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) face memorybound performance bottlenecks due to their high memory requirements. |
| Approach: | They propose a trie-based parallel decoding method that shares a single KV cache across beams with common prefixes to dramatically reduce memory usage and enables efficient decoding. |
| Outcome: | The proposed method significantly reduces memory usage and enables efficient decoding without compromising generation quality. |