Papers by Junrui Wan
AFLoRA: Adaptive Freezing of Low Rank Adaptation in Parameter Efficient Fine-Tuning of Large Models (2024.acl-short)
Copied to clipboard
| Challenge: | Pre-trained language models have demonstrated commendable performance on various NLP tasks. |
| Approach: | They propose a Parameter-Efficient Fine-Tuning (PEFT) method that incrementally freezes low-rank matrices during fine-tuning to reduce computation and alleviate over-fitting. |
| Outcome: | The proposed method achieves state-of-the-art performance with an average improvement of 0.85% on the GLUE benchmark while yielding up to 1.86 improvement as opposed to similar PEFT alternatives. |