Papers by Marjan Slavkovski
Gatekeeper to save COGS and improve efficiency of Text Prediction (2023.emnlp-industry)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) require massive amounts of computation and storage, such an approach incurs network and high execution cost. |
| Approach: | They propose a model gatekeeper to stop LLM calls that result in incorrect predictions . they show it can save 46.6% of COGS and improve user experience by not showing incorrect predictions. |
| Outcome: | The proposed model gatekeeper saves 46.6% of COGS and improves user experience . it also improves the suggestion rate of the proposed model by 73% . |