Papers by Praneeth Vepakomma
FedEx-LoRA: Exact Aggregation for Federated and Efficient Fine-Tuning of Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for low-rank averaging of LoRA adapters result in inexact updates. |
| Approach: | They propose a method which adds a residual error term to the pre-trained frozen weight matrix to achieve exact updates with minimal computational and communication overhead. |
| Outcome: | The proposed method achieves exact updates with minimal computational and communication overhead, preserving LoRA’s efficiency. |