Papers by Daria Cherniuk
Run LoRA Run: Faster and Lighter LoRA Implementations (2025.acl-industry)
Copied to clipboard
| Challenge: | Existing studies on low-rank adapter training use the default chain of operations while calculating the output. |
| Approach: | They propose a framework that allows for efficient LoRA implementations by introducing low-rank adapters to linear layers and selecting the best forward and backward graphs based on FLOPs and time estimations. |
| Outcome: | The proposed framework significantly improves the speed of neural network training and fine-tuning with low-rank adapters. |