Papers by Attreyee Mukherjee
Cache Saver: A Modular Framework for Efficient, Affordable, and Reproducible LLM Inference (2025.findings-emnlp)
Copied to clipboard
Nearchos Potamitis, Lars Henning Klein, Bardia Mohammadi, Chongyang Xu, Attreyee Mukherjee, Niket Tandon, Laurent Bindschaedler, Akhil Arora
| Challenge: | Inference is the major cost throughout the lifecycle of a large language model (LLM). |
| Approach: | They propose a plug-and-play, asynchronous framework that facilitates high-level inference optimizations. |
| Outcome: | The proposed framework reduces cost and CO2 by 35% across methods, tasks, and LLMs. |