Papers by Yair Carmon
Scaling Laws Under the Microscope: Predicting Transformer Performance from Small Scale Experiments (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Neural scaling laws define a predictable relationship between a model’s parameter count and its performance after training in the form of a power law. |
| Approach: | They perform an empirical investigation of language understanding tasks and evaluate their results to determine whether scaling laws can be used to accelerate model development. |
| Outcome: | The proposed scaling laws can be exploited for debugging convergence when training large models, and can predict the performance of larger models. |