Papers by Scott Lundberg
Fixing Model Bugs with Natural Language Patches (2022.emnlp-main)
Copied to clipboard
| Challenge: | a growing body of research focused on using language to give instructions, supervision and even inductive biases to models instead of relying exclusively on labeled examples. |
| Approach: | They explore natural language patches that provide corrective feedback at the right level of abstraction. |
| Outcome: | The proposed model improves accuracy on real data by 1–4 accuracy points on different slices of a sentiment analysis dataset and F1 by 7 points on a relation extraction dataset. |
Adaptive Testing and Debugging of NLP Models (2022.acl-long)
Copied to clipboard
| Challenge: | Current approaches to testing and debugging NLP models rely on variable human creativity and extensive labor to instantiate bugs. |
| Approach: | They propose a process which uses large scale language models to automatically write unit tests highlighting bugs in a target model. |
| Outcome: | The proposed process makes users 5-10x more effective at finding bugs than current approaches. |