Papers by Anuj Pathania
MaCP: Minimal yet Mighty Adaptation via Hierarchical Cosine Projection (2025.acl-long)
Copied to clipboard
| Challenge: | MaCP is a new adaptation method for large foundation models that requires minimal parameters and memory for fine-tuning. |
| Approach: | They propose a method that exploits the superior energy compaction and decorrelation properties of cosine projection to improve model efficiency and accuracy. |
| Outcome: | The proposed method improves model efficiency and accuracy across a wide range of single-modality tasks including natural language understanding, natural language generation, text summarization, and multi-modalities such as image classification and video understanding. |
SSH: Sparse Spectrum Adaptation via Discrete Hartley Transformation (2025.naacl-long)
Copied to clipboard
| Challenge: | Low-rank adaptation (LoRA) has been demonstrated effective in reducing the trainable parameter number when fine-tuning a large foundation model (LLM). |
| Approach: | They propose a low-rank adaptation approach that reduces the number of trainable parameters while enhancing model performance. |
| Outcome: | The proposed approach outperforms existing parameter-efficient fine-tuning methods while achieving substantial reductions in computational cost and memory requirements. |