Papers by Ekaterina Grishina
ProcrustesGPT: Compressing LLMs with Structured Matrices and Orthogonal Transformations (2025.findings-acl)
Copied to clipboard
| Challenge: | Structured matrix representations of large language models are a promising way to reduce the number of parameters in natural language processing tasks but require a significant amount of computational and memory resources. |
| Approach: | They propose to utilize invariance of the network output under certain orthogonal transformations of weight matrices to identify transformations that improve compressibility of weights within structured classes. |
| Outcome: | The proposed approach is applicable to various types of structured matrices that support efficient projection operations. |