Papers by Pranav Sharma
How to Select One Among All ? An Empirical Study Towards the Robustness of Knowledge Distillation in Natural Language Understanding (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Knowledge Distillation (KD) is a model compression algorithm that helps transfer knowledge in a large neural network into a smaller one. |
| Approach: | They propose a framework to assess adversarial robustness of multiple KD algorithms. |
| Outcome: | The proposed algorithm achieves state-of-the-art on the GLUE benchmark and out-of domain generalization and adversarial robustness compared to competitive methods. |
Annealing Knowledge Distillation (2021.eacl-main)
Copied to clipboard
| Challenge: | Knowledge distillation (KD) is a powerful model compression technique for deep neural networks. |
| Approach: | They propose a method to feed the rich information provided by teacher’s soft-targets incrementally and more efficiently by annealing the teacher output incrementally. |
| Outcome: | The proposed method can be used on image classification and NLP language inference tasks with BERT-based models on the GLUE benchmark. |
Pro-KD: Progressive Distillation by Following the Footsteps of the Teacher (2022.coling-1)
Copied to clipboard
| Challenge: | Knowledge distillation (KD) is a powerful tool for deep learning applications. |
| Approach: | They propose a method which defines a smoother training path for the student by following the training footprints of the teacher rather than solely relying on distilling from a single mature fully-trained teacher. |
| Outcome: | The proposed technique is quite effective in mitigating the capacity-gap problem and the checkpoint search problem. |
Exploring the Boundaries of GPT-4 in Radiology (2023.emnlp-main)
Copied to clipboard
Qianchu Liu, Stephanie Hyland, Shruthi Bannur, Kenza Bouzid, Daniel Castro, Maria Wetscherek, Robert Tinn, Harshita Sharma, Fernando Pérez-García, Anton Schwaighofer, Pranav Rajpurkar, Sameer Khanna, Hoifung Poon, Naoto Usuyama, Anja Thieme, Aditya Nori, Matthew Lungren, Ozan Oktay, Javier Alvarez-Valle
| Challenge: | Recent success of general-domain large language models has changed the natural language processing paradigm towards a unified foundation model across domains and applications. |
| Approach: | They evaluate the performance of GPT-4 on a variety of radiology tasks . they find it outperforms or matches current SOTA radiology models . |
| Outcome: | The proposed model outperforms or matches current SOTA radiology models on a range of tasks. |