Papers by Juan Munoz
SQFT: Low-cost Model Adaptation in Low-precision Sparse Foundation Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large pre-trained models are often adapted to a desired domain or task through a fine-tuning stage. |
| Approach: | They propose an end-to-end solution for sparse parameter-efficient fine-tuning of large pre-trained models. |
| Outcome: | The proposed approach can be used to combine sparse weights with low-rank adapters without losing sparsity and accuracy. |