Papers by Helen Margetts
HateCheck: Functional Tests for Hate Speech Detection Models (2021.acl-long)
Copied to clipboard
| Challenge: | Hate speech detection models are evaluated by measuring their performance on held-out test data using metrics such as accuracy and F1 score. |
| Approach: | They propose a suite of functional tests for hate speech detection models that measure model performance on held-out test data and then craft test cases to validate their quality. |
| Outcome: | The proposed tests show that the proposed models perform poorly on a small set of widely-used hate speech datasets. |
An Expert Annotated Dataset for the Detection of Online Misogyny (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing studies have found that misogynistic content is pervasive on some Reddit communities, but a training dataset for misogorical classification has not been created with the data. |
| Approach: | They propose a hierarchical taxonomy and an expert labelled dataset to enable automatic classification of online misogynistic content. |
| Outcome: | The proposed taxonomy and an expert labelled dataset are made freely available for future research. |
Introducing CAD: the Contextual Abuse Dataset (2021.naacl-main)
Copied to clipboard
| Challenge: | Detecting and classifying online abuse is a complex and nuanced task, despite many advances in the power and availability of computational tools. |
| Approach: | They propose to annotate a reddit conversation thread with six distinct primary and secondary categories and an expert-driven group-adjudication process for high quality annotations. |
| Outcome: | The proposed dataset contains six distinct primary and secondary categories and uses an expert-driven group-adjudication process for high quality annotations. |