Papers by Vladimir Solodkin

Copied to clipboard

Challenge:	Low-rank adaptation (LoRA) adds trainable adapters to selected layers, but requires significant memory to train large models and intuition on which layers to add adapters.
Approach:	They propose a method which adds trainable adapters to selected layers . they compare weightLoRA with different adaptive approaches to reduce trainable parameters while maintaining consistent or even superior metric values.
Outcome:	The proposed method reduces the number of trainable parameters while maintaining the capability to obtain consistent or even superior metric values.