Papers by Raviteja Vemulapalli
MUSCLE: A Model Update Strategy for Compatible LLM Evolution (2024.findings-emnlp)
Copied to clipboard
Jessica Echterhoff, Fartash Faghri, Raviteja Vemulapalli, Ting-Yao Hu, Chun-Liang Li, Oncel Tuzel, Hadi Pouransari
| Challenge: | Large Language Models (LLMs) are regularly updated to enhance performance, typically through changes in data or architecture. |
| Approach: | They propose a method to minimize the extent of instance regression in model updates by training a compatibility adapter that can enhance task fine-tuned language models. |
| Outcome: | The proposed approach reduces regressions by up to 40% when updating LLMs to newer versions while maintaining overall performance gains. |
TiC-LM: A Web-Scale Benchmark for Time-Continual LLM Pretraining (2025.acl-long)
Copied to clipboard
Jeffrey Li, Mohammadreza Armandpour, Seyed Iman Mirzadeh, Sachin Mehta, Vaishaal Shankar, Raviteja Vemulapalli, Samy Bengio, Oncel Tuzel, Mehrdad Farajtabar, Hadi Pouransari, Fartash Faghri
| Challenge: | Large language models (LLMs) trained on historical web data inevitably become outdated. |
| Approach: | They propose a web-scale dataset for time-continual pretraining of LLMs derived from 114 dumps of Common Crawl (CC) they also design time-stratified evaluations to assess how well various continual learning methods adapt to new data while retaining past knowledge. |
| Outcome: | The proposed benchmarks show that autoregressive meta-schedules combined with a fixed-ratio replay of older data can achieve comparable held-out loss to re-training from scratch, while requiring significantly less computation (2.6x). |
Mutual Reinforcement of LLM Dialogue Synthesis and Summarization Capabilities for Few-Shot Dialogue Summarization (2025.findings-naacl)
Copied to clipboard
Yen-Ju Lu, Ting-Yao Hu, Hema Swetha Koppula, Hadi Pouransari, Jen-Hao Rick Chang, Yin Xia, Xiang Kong, Qi Zhu, Xiaoming Simon Wang, Oncel Tuzel, Raviteja Vemulapalli
| Challenge: | Empirical results demonstrate that our method improves dialogue summarization, achieving a 1.5% increase in ROUGE scores and a 0.3% improvement in BERT scores in few-shot settings. |
| Approach: | They propose Mutual Reinforcing Data Synthesis (MRDS) within large language models to enhance few-shot dialogue summarization task. |
| Outcome: | Empirical results show that the proposed method improves dialogue summarization, achieving a 1.5% increase in ROUGE scores and a 0.3% improvement in BERT scores in few-shot settings. |