Papers by Jaewon Cheon
COUNTDOWN: Contextually Sparse Activation Filtering Out Unnecessary Weights in Down Projection (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models have created significant computational inefficiencies due to their size and complexity. |
| Approach: | They propose to use a linear combination to deactivate non-essential parameters during inference to reduce computational costs. |
| Outcome: | The proposed methods can omit 90% of computations with performance loss as low as 5.5% ideally, while M-COUNTDOWN provides a predictor-free solution with up to 29.4% better performance preservation compared to existing methods. |