Papers by Supriyo Chakraborty
Training Dynamics Underlying Language Model Scaling Laws: Loss Deceleration and Zero-Sum Learning (2025.acl-long)
Copied to clipboard
| Challenge: | Increasing language model size improves cross-entropy loss with power-law behaviour, but scaling laws do not explain how scaling improves loss. |
| Approach: | They find that language models undergo loss deceleration early in training . they attribute loss deceleration to a type of degenerate training dynamics we call zero-sum learning . |
| Outcome: | The proposed scaling improves loss on language models, but degrades loss in other subsets, resulting in bottlenecks. |
Routing with Generated Data: Annotation-Free LLM Skill Estimation and Expert Selection (2026.acl-long)
Copied to clipboard
Tianyi Niu, Justin Chen, Genta Indra Winata, Shi-Xiong Zhang, Supriyo Chakraborty, Sambit Sahu, Yue Zhang, Elias Stengel-Eskin, Mohit Bansal
| Challenge: | Existing approaches typically assume access to ground-truth labeled data . Existing methods require a classifier to select models given an input . |
| Approach: | They propose a routing setting where routers are trained exclusively on generated queries and answers from LLMs. |
| Outcome: | The proposed router outperforms the best query-answer router by 4.6% absolute accuracy when trained on weak generator data. |