Papers by Prajjwal Bhargava
DiscoSense: Commonsense Reasoning with Discourse Connectives (2022.emnlp-main)
Copied to clipboard
| Challenge: | DiscoSense is a benchmark for commonsense reasoning using a wide variety of discourse connectives. |
| Approach: | They propose a benchmark for commonsense reasoning by understanding a wide variety of discourse connectives. |
| Outcome: | The proposed benchmark outperforms existing benchmarks on commonsense reasoning tasks. |
BTS: Harmonizing Specialized Experts into a Generalist LLM (2025.emnlp-main)
Copied to clipboard
Qizhen Zhang, Prajjwal Bhargava, Chloe Bi, Chris X. Cai, Jakob Nicolaus Foerster, Jeremy Fu, Punit Singh Koura, Ruan Silva, Sheng Shen, Emily Dinan, Suchin Gururangan, Mike Lewis
| Challenge: | Branch-Train-Stitch (BTS) is an efficient and flexible training algorithm for combining independently trained large language model (LLM) experts into a single, capable generalist model. |
| Approach: | They propose an efficient and flexible training algorithm for combining large language model (LLM) experts into a single, capable generalist model using lightweight stitch layers. |
| Outcome: | The proposed model can generalize to new domains despite being frozen . it yields the best generalist performance on a variety of downstream tasks, retaining the specialized capabilities of each of the experts. |
Effective Long-Context Scaling of Foundation Models (2024.naacl-long)
Copied to clipboard
Wenhan Xiong, Jingyu Liu, Igor Molybog, Hejia Zhang, Prajjwal Bhargava, Rui Hou, Louis Martin, Rashi Rungta, Karthik Abinav Sankararaman, Barlas Oguz, Madian Khabsa, Han Fang, Yashar Mehdad, Sharan Narang, Kshitiz Malik, Angela Fan, Shruti Bhosale, Sergey Edunov, Mike Lewis, Sinong Wang, Hao Ma
| Challenge: | Large language models (LLMs) are rapidly deployed and continue to evolve through scaling. |
| Approach: | They propose a method to train strong long-context LLMs that are capable of utilizing massive context windows of up to 32,000 tokens. |
| Outcome: | The proposed model can surpass gpt-3.5-turbo-16k's overall performance on long-context benchmarks with a cost-effective instruction tuning procedure that is free of expensive annotations. |
Adaptive Transformers for Learning Multimodal Representations (2020.acl-srw)
Copied to clipboard
| Challenge: | Existing approaches for learning visiolinguistic representations with transformers are over-parametrized and require extensive training. |
| Approach: | They propose to extend attention spans, sparse, and structured dropout methods to learn more about how the network perceives the complexity of input sequences. |
| Outcome: | The proposed approaches improve on language semantics and visiolinguistic representations, but are often over-parametrized and require large amounts of computation. |