Papers by Vijeta Deshpande
Emergent Abilities in Reduced-Scale Generative Language Models (2024.findings-naacl)
Copied to clipboard
| Challenge: | Large language models can solve new tasks without task-specific fine-tuning. |
| Approach: | They propose to use pre-training data to pre-train 36 language models with billions of parameters to investigate whether emergent properties are tied to model size or can be demonstrated by smaller models. |
| Outcome: | The proposed model performs comparable to models trained on unrestricted language. |
Honey, I Shrunk the Language: Language Model Behavior at Reduced Scale. (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have focused on high-compute settings, leaving the question of when these abilities begin to emerge largely unanswered. |
| Approach: | They investigate whether effects of pre-training can be observed when problem size is reduced, modeling a smaller, reduced-vocabulary language. |
| Outcome: | The proposed model performance is correlated with pre-training perplexity and performance. |
Diverse, not Short: A Length-Controlled Data Selection Strategy for Improving Response Diversity of Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Diverse language model responses are crucial for creative generation, open-ended tasks, and self-improvement training. |
| Approach: | They propose a length-controlled data selection strategy that improves diversity while maintaining length parity. |
| Outcome: | The proposed method improves diversity while maintaining length parity on LLaMA-3.1-8B and Olmo-2 family. |
LocalTweets to LocalHealth: A Mental Health Surveillance Framework Based on Twitter Data (2024.lrec-main)
Copied to clipboard
| Challenge: | Prior research on Twitter has provided positive evidence of its utility in developing supplementary health surveillance systems. |
| Approach: | They propose a framework to surveil public health, focusing on mental health outcomes by using tweets from 765 neighborhoods in the USA. |
| Outcome: | The proposed framework achieves the highest F1-score and accuracy over the previous framework, and extrapolates CDC’s estimates to proxy unreported neighborhoods. |