Papers by Prateek Yadav
Explanation Graph Generation via Pre-trained Language Models: An Empirical Study with Contrastive Learning (2022.acl-long)
Copied to clipboard
| Challenge: | Pre-trained sequence-to-sequence language models generate structured outputs such as graphs with limited supervision. |
| Approach: | They propose to use pre-trained sequence-to-sequence language models to generate graphs . they propose to learn structural constraints and semantics of graphs with limited supervision . |
| Outcome: | The proposed models can learn structural constraints and semantics of graphs with limited supervision. |
Aurora-M: Open Source Continual Pre-training for Multilingual Language and Code (2025.coling-industry)
Copied to clipboard
Taishi Nakamura, Mayank Mishra, Simone Tedeschi, Yekun Chai, Jason T. Stillerman, Felix Friedrich, Prateek Yadav, Tanmay Laud, Vu Minh Chien, Terry Yue Zhuo, Diganta Misra, Ben Bogin, Xuan-Son Vu, Marzena Karpinska, Arnav Varma Dantuluri, Wojciech Kusa, Tommaso Furlanello, Rio Yokota, Niklas Muennighoff, Suhas Pai, Tosin Adewumi, Veronika Laippala, Xiaozhe Yao, Adalberto Barbosa Junior, Aleksandr Drozd, Jordan Clive, Kshitij Gupta, Liangyu Chen, Qi Sun, Ken Tsui, Nour Moustafa-Fahmy, Nicolo Monti, Tai Dang, Ziyang Luo, Tien-Tung Bui, Roberto Navigli, Virendra Mehta, Matthew Blumberg, Victor May, Hiep Nguyen, Sampo Pyysalo
| Challenge: | Pretrained language models are integral part of AI applications, but their high computational cost limits accessibility. |
| Approach: | They evaluate Aurora-M, a 15B parameter multilingual open-source model trained on English, Finnish, Hindi, Japanese, Vietnamese, and code. |
| Outcome: | The proposed model outperforms existing models on English, Finnish, Hindi, Japanese, Vietnamese, and code. |
ExplaGraphs: An Explanation Graph Generation Task for Structured Commonsense Reasoning (2021.emnlp-main)
Copied to clipboard
| Challenge: | Current commonsense-reasoning tasks are discriminative in nature, where a model answers a multiple-choice question for a certain context. |
| Approach: | They propose a generative task that generates a commonsense-augmented graph for stance prediction by using a create-verify-and-refine graph collection framework. |
| Outcome: | The proposed model is able to generate a graph that serves as non-trivial, complete, and unambiguous explanation for the predicted stance. |
Incorporating Syntactic and Semantic Information in Word Embeddings using Graph Convolutional Networks (P19-1)
Copied to clipboard
Shikhar Vashishth, Manik Bhandari, Prateek Yadav, Piyush Rai, Chiranjib Bhattacharyya, Partha Talukdar
| Challenge: | Existing word embedding methods utilize sequential context of a word to learn its embeddment, but such methods result in an explosion of the vocabulary size. |
| Approach: | They propose a flexible Graph Convolution based method for learning word embeddings that utilizes the dependency context of a word without increasing the vocabulary size. |
| Outcome: | The proposed model outperforms existing methods on intrinsic and extrinsic tasks and provides an advantage when used with ELMo. |
Exploring Continual Learning for Code Generation Models (2023.acl-short)
Copied to clipboard
Prateek Yadav, Qing Sun, Hantian Ding, Xiaopeng Li, Dejiao Zhang, Ming Tan, Parminder Bhatia, Xiaofei Ma, Ramesh Nallapati, Murali Krishna Ramanathan, Mohit Bansal, Bing Xiang
| Challenge: | Large-scale code generation models such as Copilot and CodeT5 are expensive to train and re-train. |
| Approach: | They propose a benchmark for Continual Learning (CL) that covers a wide range of tasks with different input and output programming languages. |
| Outcome: | The proposed method improves on Prompt Pooling with Teacher Forcing, which suffers catastrophic forgetting due to stark distribution shifts in coding tasks. |
Exclusive Supermask Subnetwork Training for Continual Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | Continual Learning (CL) methods focus on accumulating knowledge over time while preventing catastrophic forgetting. |
| Approach: | They propose a CL method that finds a supermask for each new task that keeps or removes each weight to produce a subnetwork. |
| Outcome: | The proposed method outperforms strong previous methods on NLP and Vision domains while preventing forgetting. |
multiPRover: Generating Multiple Proofs for Improved Interpretability in Rule Reasoning (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing work to generate proof graphs for formal reasoning over explicit knowledge is not unique and there may be multiple ways of reaching the correct answer. |
| Approach: | They propose to generate multiple proof graphs for reasoning over natural language rules and facts . they propose to combine all proofs and exploit correlations between them . |
| Outcome: | The proposed model outperforms PRover on multiple gold proofs on synthetic, zero-shot, and human-paraphrased datasets. |
Glider: Global and Local Instruction-Driven Expert Router (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for routing-based expert models favor generalization over performance on held-in tasks. |
| Approach: | They propose a global and local instruction driven expert router that leverages recent LLMs' semantic reasoning capabilities to generate task-specific instructions from the input query. |
| Outcome: | The proposed method improves held-in performance while maintaining strong generalization on held-out tasks. |