Papers by Peter Mattson
Dynatask: A Framework for Creating Dynamic AI Benchmark Tasks (2022.acl-demo)
Copied to clipboard
Tristan Thrush, Kushal Tirumala, Anmol Gupta, Max Bartolo, Pedro Rodriguez, Tariq Kane, William Gaviria Rojas, Peter Mattson, Adina Williams, Douwe Kiela
| Challenge: | Open source system for setting up custom NLP tasks aims to lower technical knowledge and effort required for hosting and evaluating state-of-the-art models. |
| Approach: | They propose to integrate Dynatask with Dynabench to simplify benchmarking . they use a dataset to collect and clean data and train and evaluate models . |
| Outcome: | Dynatask is an open source system for setting up custom NLP tasks . it is integrated with Dynabench, a research platform for rethinking benchmarking in AI . |