Papers by Yara Shamshoum
CompAct: Compressed Activations for Memory-Efficient LLM Training (2025.naacl-long)
Copied to clipboard
| Challenge: | Recent studies have focused on reducing peak memory utilization on GPUs, but most work only target the computation graph during training. |
| Approach: | They propose a technique that reduces peak memory utilization on GPUs by 25-30% for pretraining and 50% for fine-tuning of LLMs. |
| Outcome: | The proposed technique reduces peak memory utilization on GPUs by 25-30% for pretraining and 50% for fine-tuning of LLMs. |