Papers by Seongbae Lee
Riemannian Optimization for LoRA on the Stiefel Manifold (2025.findings-emnlp)
Copied to clipboard
| Challenge: | powerful, large language models (LLMs) present significant fine-tuning challenges due to their size. |
| Approach: | They propose to optimize LoRA’s B matrix by imposing explicit orthogonality constraints that achieve near-perfect orthogonal and full effective rank. |
| Outcome: | The proposed method outperforms AdamW and LoRA in terms of parameter efficiency and representational capacity. |