Papers by Lisa Haverty
Iterative Stratified Testing and Measurement for Automated Model Updates (2022.emnlp-industry)
Copied to clipboard
Elizabeth Dekeyser, Nicholas Comment, Shermin Pei, Rajat Kumar, Shruti Rai, Fengtao Wu, Lisa Haverty, Kanna Shimizu
| Challenge: | Automating updates to machine learning systems is an important but understudied challenge in AutoML. |
| Approach: | They propose a framework that relies on iterative model building coupled with data-shape stratified model testing and improvement to improve model accuracy. |
| Outcome: | The proposed framework shows a 26% improvement in accuracy for new model use cases on a large-scale NLU system compared to a naive baseline and current cutting-edge methods. |