Papers by Frederic Sala
The Cost of Compression: Investigating the Impact of Compression on Parametric Knowledge in Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing research on LLM compression focuses on general metrics like perplexity or downstream task accuracy. |
| Approach: | They propose to quantify the effect of pruning and quantization on model quality . they use the LAMA and LM-Harness benchmarks to quantify compression techniques . |
| Outcome: | The proposed compression techniques provide faster inference, smaller memory footprints, and enables local deployment. |
Low-Dimensional Hyperbolic Knowledge Graph Embeddings (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods for predicting missing facts do not account for hierarchical and logical patterns in KGs. |
| Approach: | They propose a class of hyperbolic KG embedding models that capture hierarchical and logical patterns. |
| Outcome: | Experimental results show that the proposed method improves by 6.1% in mean reciprocal rank in low dimensions over previous methods. |
Personalize Your LLM: Fake it then Align it (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing personalization methods require fine-tuning of large language models for each user, rendering them prohibitively expensive for widespread adoption. |
| Approach: | They propose a retrieval-based personalization approach that uses self-generated personal preference data and representation editing to enable quick and cost-effective personalization. |
| Outcome: | The proposed approach outperforms two personalization baselines by 40% on various tasks. |
Look Who’s Talking Now: Covert Channels From Biased LLMs (2024.findings-emnlp)
Copied to clipboard
| Challenge: | steganography encodes hidden messages into model-generated tokens . tradeoff between how much hidden information can be introduced and how much the model can be perturbed is important . |
| Approach: | They propose to use large language model-based steganography to encode hidden messages into model-generated tokens. |
| Outcome: | The proposed techniques are nearly optimal under a practical but difficult set of constraints . the proposed techniques ensure that only someone with the appropriate decoding key can access the hidden information . |