Papers by Owen Cook
Efficient Annotator Reliability Assessment and Sample Weighting for Knowledge-Based Misinformation Detection on Social Media (2025.findings-naacl)
Copied to clipboard
Owen Cook, Charlie Grimshaw, Ben Peng Wu, Sophie Dillon, Jack Hicks, Luke Jones, Thomas Smith, Matyas Szert, Xingyi Song
| Challenge: | Misinformation spreads rapidly on social media, confusing the truth and targeting potentially vulnerable people. |
| Approach: | They propose to use inter- and intra-annotator agreement to understand the reliability of each annotator and influence the training of large language models based on annotators reliability. |
| Outcome: | The proposed framework utilises inter- and intra-annotator agreement to understand the reliability of each annotator and influence the training of large language models based on annotators reliability. |
Efficient Annotator Reliability Assessment with EffiARA (2025.acl-demo)
Copied to clipboard
| Challenge: | Obtaining annotations from experts is ideal, but this expertise is logistically and financially costly. |
| Approach: | They propose an annotation framework that supports the whole annotation pipeline from understanding the resources required for an annotation task to compiling the annotated dataset. |
| Outcome: | The proposed framework improves classification performance through annotator-reliability-based soft-label aggregation and sample weighting, and increases agreement among annotators through removal of identifying and replacing an unreliable annotation. |