Papers by Ganesh Nathan
Wanda++: Pruning Large Language Models via Regional Gradients (2025.findings-acl)
Copied to clipboard
Yifan Yang, Kai Zhen, Bhavana Ganesh, Aram Galstyan, Goeric Huybrechts, Markus Müller, Jonas M. Kübler, Rupak Vignesh Swaminathan, Athanasios Mouchtaris, Sravan Babu Bodapati, Nathan Susanj, Zheng Zhang, Jack FitzGerald, Abhishek Kumar
| Challenge: | Existing pruning methods suffer from accuracy degradation without full-model sparsity-aware fine-tuning. |
| Approach: | They propose a pruning framework that uses decoder-block-level regional gradients to improve pruning accuracy. |
| Outcome: | The proposed pruning framework outperforms the state-of-the-art pruning frameworks by utilizing decoder-block-level regional gradients. |
Fisher Mask Nodes for Language Model Merging (2024.lrec-main)
Copied to clipboard
| Challenge: | Pre-trained models are ubiquitous in natural language processing, but individual fine-tuned models require significant overhead in multi-task scenarios. |
| Approach: | They propose a method for fine-tuning pre-trained models for Transformers using Fisher information. |
| Outcome: | The proposed method outperforms Fisher-weighted averaging in a fraction of the computational cost. |