Papers by She Yifei
DisLoRA: Task-specific Low-Rank Adaptation via Orthogonal Basis from Singular Value Decomposition (2025.emnlp-main)
Copied to clipboard
| Challenge: | Extensive experiments on GLUE and Commonsense Reasoning benchmarks demonstrate that DisLoRA surpasses established PEFT methods, including LoRA, PiSSA, DoRA, LoRA-Dash, and SORSA. |
| Approach: | They propose a framework that leverages singular value decomposition to decompose pretrained weight matrices into orthogonal backbone and task-specific subspaces. |
| Outcome: | Extensive experiments on GLUE and Commonsense Reasoning benchmarks show that DisLoRA surpasses established PEFT methods, including LoRA, PiSSA, DoRA, LoRA-Dash, and SORSA. |