Papers by Yuri Hong
Memory-Efficient Structured Backpropagation for On-Device LLM Fine-Tuning (2026.acl-industry)
Copied to clipboard
| Challenge: | Existing approaches for fine-tuning large language models require a trade-off between exact gradients with high memory and low memory with noisy estimates (MeZO). |
| Approach: | They propose a method which derivates gradients from LoRA's low-rank structure and manually deriving backward passes to exploit the low-level structure. |
| Outcome: | The proposed method reduces peak memory from 361MB to 136MB for Qwen2.5-0.5B, enabling fine-tuning scenarios previously infeasible on memory-constrained devices. |