Papers by Rajesh Ranganath
Learning Is Not A Race: Improving Retrieval in Language Models via Equal Learning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Overparametrized models trained on cross-entropy loss can overfit on noise . Fitting some tokens early reduces gradient signals in later iterations . |
| Approach: | They propose to overfit models trained on cross-entropy loss on noise . fitting some tokens early reduces gradient signals in later iterations . |
| Outcome: | The proposed approaches can be applied to large language models with longer contexts or larger embedding sizes. |