Papers by Divyaksh Shukla
Towards Robust Evaluation of Unlearning in LLMs via Data Transformations (2024.findings-emnlp)
Copied to clipboard
Abhinav Joshi, Shaswati Saha, Divyaksh Shukla, Sriram Vema, Harsh Jhamtani, Manas Gaur, Ashutosh Modi
| Challenge: | Large Language Models (LLMs) have shown to be a great success in a wide range of applications ranging from regular NLP-based use cases to AI agents. |
| Approach: | They examine the robustness of existing MUL techniques for their ability to enable leakage-proof forgetting in LLMs. |
| Outcome: | The proposed methods can be used to enable leakage-proof forgetting in LLMs. |
Towards Quantifying Commonsense Reasoning with Mechanistic Insights (2025.naacl-long)
Copied to clipboard
| Challenge: | Recent studies have evaluated commonsense reasoning abilities using text-based tasks. |
| Approach: | They propose to capture commonsense knowledge in a graphical representation of 37 daily human activities in graphical form and frame them to frame commonsensical queries. |
| Outcome: | The proposed model can frame an enormous number of commonsense queries ( 10 17) and perform rigorous evaluations of common sense reasoning in LLMs. |
Calibration vs Decision Making: Revisiting the Reliability Paradox in Unlearned Language Models (2026.acl-srw)
Copied to clipboard
| Challenge: | Model calibration is commonly used as a proxy for reliability, but low calibration error does not necessarily imply reliable decision rules. |
| Approach: | They investigate the calibration error gap in generative language models using the TOFU benchmark and attribution-based shortcut detection. |
| Outcome: | The proposed model calibrations achieve low calibration error compared to pretrained models and retain low calibration despite reduced accuracy on the forget split. |
CoMuMDR: Code-mixed Multi-modal Multi-domain corpus for Discourse paRsing in conversations (2025.findings-acl)
Copied to clipboard
| Challenge: | Discourse parsing datasets based on conversations are restricted to a single domain . a lack of discourse structures in audio-based conversations is a challenge . |
| Approach: | They introduce CoMuMDR: Code-mixed Multi-modal Multi-domain corpus for Discourse parsing in conversations. |
| Outcome: | The proposed corpus is code-mixed in Hindi and English and annotated with nine discourse relations. |