Papers by Ashiqur KhudaBukhsh
Subjective Crowd Disagreements for Subjective Data: Uncovering Meaningful CrowdOpinion with Population-level Learning (2023.acl-long)
Copied to clipboard
| Challenge: | Annotator disagreements are resolved before learning takes place, but researchers question the performance of a system when annotators disagree. |
| Approach: | They propose a method that uses language features and label distributions to pool similar items into larger labels. |
| Outcome: | The proposed method is based on five publicly available datasets with varying levels of disagreements on social media and in the wild using a dataset from Facebook. |
Vicarious Offense and Noise Audit of Offensive Speech Classifiers: Unifying Human and Machine Disagreement on What is Offensive (2023.emnlp-main)
Copied to clipboard
Tharindu Weerasooriya, Sujan Dutta, Tharindu Ranasinghe, Marcos Zampieri, Christopher Homan, Ashiqur KhudaBukhsh
| Challenge: | a paper examines how machine and human moderators disagree on offensive speech . offensive speech detection is a key component of content moderation . |
| Approach: | They propose a large-scale noise audit and a vicarious offense dataset to investigate disagreement on social web political discourse. |
| Outcome: | The proposed dataset reveals that moderation outcomes vary wildly across different machine moderators. |
Disagreement Matters: Preserving Label Diversity by Jointly Modeling Item and Annotator Label Distributions with DisCo (2023.findings-acl)
Copied to clipboard
Tharindu Cyril Weerasooriya, Alexander Ororbia, Raj Bhensadadia, Ashiqur KhudaBukhsh, Christopher Homan
| Challenge: | a recent study shows that annotator disagreement is common in supervised learning . a simple neural model that learns to predict annotators' labels is competitive with other models that do not model specific annotations. |
| Approach: | They propose a neural model that learns to predict annotator distributions by aggregating over all annotators. |
| Outcome: | The proposed model outperforms models that do not model specific annotators or do not learn label distribution learning. |
Social Media Attributions in the Context of Water Crisis (2020.emnlp-main)
Copied to clipboard
| Challenge: | In this paper, we analyze social media discussions to identify attribution factors for natural disasters/collective misfortunes. |
| Approach: | They propose a task of attribution tie detection to identify factors held responsible for a water crisis in a social media document. |
| Outcome: | The proposed task can be performed on a dataset constructed from YouTube comments on 2,500 videos relevant to the 2019 Chennai water crisis. |
Rater Cohesion and Quality from a Vicarious Perspective (2024.findings-emnlp)
Copied to clipboard
Deepak Pandita, Tharindu Cyril Weerasooriya, Sujan Dutta, Sarah Luger, Tharindu Ranasinghe, Ashiqur KhudaBukhsh, Marcos Zampieri, Christopher Homan
| Challenge: | Recent work in reinforcement learning with human feedback (RLHF) highlights the gains in model performance from aligning them to human values. |
| Approach: | They propose to use vicarious annotation to break down disagreement by asking raters how they think others would annotate the data. |
| Outcome: | The proposed method breaks down disagreements by asking raters how they think others would annotate the data. |