Papers by Brian Riordan
Don’t take “nswvtnvakgxpm” for an answer –The surprising vulnerability of automatic content scoring systems to adversarial input (2020.coling-main)
Copied to clipboard
| Challenge: | Automated content scoring systems can be used on short answer tasks to save human effort, but can invite cheating strategies such as writing irrelevant answers. |
| Approach: | They generate adversarial answers for benchmark content scoring datasets based on different methods of increasing sophistication and examine countermeasures such as adversarials. |
| Outcome: | The proposed methods show that even simple methods can reduce content scoring performance but do not solve the problem. |
Atypical Inputs in Educational Applications (N18-3)
Copied to clipboard
| Challenge: | atypical characteristics of some responses make it difficult for an automated scoring system to assign a valid score . a typical spoken response with a lot of background noise may suffer from frequent errors in automated speech recognition . |
| Approach: | They propose a pipeline that detects and processes non-scorable responses at run-time . they also propose linguistic filtering models for spoken responses in language tests . |
| Outcome: | The proposed pipeline detects and processes non-scorable responses at run-time and evaluates them for spoken responses in language proficiency assessment. |