Papers by Marco Molinari
Group-SAE: Efficient Training of Sparse Autoencoders for Large Language Models via Layer Groups (2025.emnlp-main)
Copied to clipboard
| Challenge: | Sparse Autoencoders (SAEs) are a promising unsupervised approach for understanding the representations of layers of Large Language Models (LLMs). |
| Approach: | They propose a method that groups similar models and trains a single SAE per group based on representational similarity across layers. |
| Outcome: | Experiments on Pythia family models show that the proposed method significantly accelerates training with minimal impact on reconstruction quality and comparable downstream task performance and interpretability over baseline SAEs trained layer by layer. |
Interpretable Company Similarity with Sparse Autoencoders (2025.acl-industry)
Copied to clipboard
Marco Molinari, Victor Shao, Luca Imeneo, Mateusz Mikolajczak, Abhimanyu Pandey, Vladimir Tregubiak, Sebastião Kuznetsov Ryder Torres Pereira
| Challenge: | Traditionally, company comparisons rely on relative returns and discrete classifications, or a combination of both. |
| Approach: | They propose to use clusters of embeddings to enhance the interpretability of Large Language Models by decomposing Large Language models activations into interpretable features. |
| Outcome: | The proposed clusters of embeddings capture the internal representation of a company description, rather than just semantic similarity alone. |