Papers by Alastair Roberts
Beat the AI: Investigating Adversarial Human Annotation for Reading Comprehension (2020.tacl-1)
Copied to clipboard
| Challenge: | Innovations in annotation methodologies have been a catalyst for Reading Comprehension (RC) datasets and models. |
| Approach: | They propose to use a model-in-the-annotation-loop approach to train adversarial models in three different settings to explore reproducibility of the adversarial effect, transfer from data collected with varying model- in-the loop strengths, and generalization to data collected without a modeling model. |
| Outcome: | The proposed approach achieves 39.9F1 on questions it cannot answer when trained on SQUAD, but lower than when trained using RoBERTa itself (41.0F1). |