Papers by Ethan Selfridge
Two-tiered Encoder-based Hallucination Detection for Retrieval-Augmented Generation in the Wild (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Existing solutions for hallucination detection do not consider latency, train or evaluate on production data. |
| Approach: | They propose to use customer service conversation data to evaluate existing methods . they propose to train small encoder models on a new dataset to outperform existing methods. |
| Outcome: | The proposed model outperforms existing methods and highlights the value of combining small amounts of in-domain data with public datasets. |
The economic trade-offs of large language models: A case study (2023.acl-industry)
Copied to clipboard
Kristen Howell, Gwen Christian, Pavel Fomitchov, Gitit Kehat, Julianne Marzulla, Leanne Rolston, Jadin Tredup, Ilana Zimmerman, Ethan Selfridge, Joseph Bradley
| Challenge: | Large Language Models (LLMs) are a natural fit for contact-based customer service, but their efficacy must be balanced with the cost of training and serving them. |
| Approach: | They propose a cost framework for evaluating an NLP model’s utility for the enterprise as a function of the usefulness of the responses that they generate. |
| Outcome: | The proposed model can be used to help human agents handle complex customer service calls and can be modified to improve their performance. |