Papers by Ryan Solgi
FLAT-LLM: Fine-grained Low-rank Activation Space Transformation for Large Language Model Compression (2026.findings-eacl)
Copied to clipboard
| Challenge: | Low-rank decomposition methods suffer from accuracy degradation and expensive calibration procedures. |
| Approach: | They propose a fast and accurate, training-free structural compression method based on fine-grained low-rank transformations in the activation space. |
| Outcome: | The proposed method outperforms pruning baselines in generalization and downstream performance while delivering inference speedups. |
Saten: Sparse Augmented Tensor Networks for Post-Training Compression of Large Language Models (2025.findings-emnlp)
Copied to clipboard
Ryan Solgi, Kai Zhen, Rupak Vignesh Swaminathan, Nathan Susanj, Athanasios Mouchtaris, Siegfried Kunzmann, Zheng Zhang
| Challenge: | Low-rank tensor compression techniques are used for over-parameterized neural networks, but their applications to compress pre-trained LLMs for downstream tasks remain challenging due to the high-rank nature of pre-training data. |
| Approach: | They propose sparse augmented tensor networks to enhance low-rank tenorized LLMs . they also propose a framework that enables full model compression . |
| Outcome: | The proposed framework improves accuracy and efficiency in tensorized language models. |