Papers by Sara Kangaslahti
Fast Forwarding Low-Rank Training (2024.emnlp-main)
Copied to clipboard
| Challenge: | Modern optimizers provide a spectacular array of tweaks to stabilize training trajectories and accelerate Stochastic Gradient Descent (SGD). |
| Approach: | They propose a fast-forward approach to accelerate large segments of SGD training . they alternate between Adam SGD for burn-in and accelerating by line search . |
| Outcome: | The proposed approach speeds up training without compromising model performance. |