Biasly: An Expert-Annotated Dataset for Subtle Misogyny Detection and Mitigation (2024.findings-acl)
Copied to clipboard
Brooklyn Sheppard, Anna Richter, Allison Cohen, Elizabeth Smith, Tamara Kneese, Carolyne Pelletier, Ioana Baldini, Yue Dong
| Challenge: | the Biasly dataset captures misogyny in movies in ways unique within the literature. |
| Approach: | The Biasly dataset captures misogyny in North American film by combining annotations of movie subtitles with common NLP algorithms. |
| Outcome: | The Biasly dataset captures misogyny expressions in North American film . it contains annotations of movie subtitles and text generation for rewrites . |
Similar Papers
An Expert Annotated Dataset for the Detection of Online Misogyny (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing studies have found that misogynistic content is pervasive on some Reddit communities, but a training dataset for misogorical classification has not been created with the data. |
| Approach: | They propose a hierarchical taxonomy and an expert labelled dataset to enable automatic classification of online misogynistic content. |
| Outcome: | The proposed taxonomy and an expert labelled dataset are made freely available for future research. |
Annotating Online Misogyny (2021.acl-long)
Copied to clipboard
| Challenge: | Online misogyny is a category of online abusive language with serious and harmful social consequences. |
| Approach: | They propose an iterative annotation process and a taxonomy of labels for annotating misogyny in natural written language and cite a high-quality dataset of annotated posts from social media posts. |
| Outcome: | The proposed method aims to identify misogynistic language in natural written language and annotate it in social media posts using a high-quality dataset. |
Beyond Dataset Creation: Critical View of Annotation Variation and Bias Probing of a Dataset for Online Radical Content Detection (2025.coling-main)
Copied to clipboard
| Challenge: | Existing datasets and models fail to address the complexities of multilingual data, authors say . detection of radical content on online platforms has become an increasingly pressing concern . |
| Approach: | They propose a publicly available multilingual dataset annotated with radicalization levels, calls for action, and named entities in English, French, and Arabic. |
| Outcome: | The proposed dataset is annotated with radicalization levels, calls for action, and named entities in English, French, and Arabic. |
Hollywood Identity Bias Dataset: A Context Oriented Bias Analysis of Movie Dialogues (2022.lrec-1)
Copied to clipboard
Sandhya Singh, Prapti Roy, Nihar Sahoo, Niteesh Mallela, Himanshu Gupta, Pushpak Bhattacharyya, Milind Savagaonkar, Nidhi Sultan, Roshni Ramnani, Anutosh Maitra, Shubhashis Sengupta
| Challenge: | Movies reflect society and also hold power to transform opinions. |
| Approach: | They propose to annotate movie scripts for identity bias using a dataset that is annotated for gender, race/ethnicity, religion, age, occupation, LGBTQ, and other . |
| Outcome: | The proposed dataset contains dialogue turns annotated for gender, race/ethnicity, religion, age, occupation, LGBTQ, and other, which contains biases like body shaming, personality bias, etc. |
NLPositionality: Characterizing Design Biases of Datasets and Models (2023.acl-long)
Copied to clipboard
| Challenge: | Design biases in NLP systems often stem from creator’s positionality, i.e., views and lived experiences shaped by identity and background. |
| Approach: | They propose a framework for characterizing design biases and quantifying the positionality of NLP datasets and models. |
| Outcome: | The proposed framework characterizes design biases and quantifies alignment with dataset labels and model predictions. |
From Prejudice to Parity: A New Approach to Debiasing Large Language Model Word Embeddings (2025.coling-main)
Copied to clipboard
| Challenge: | Existing work in this field has looked most commonly into gender bias, racial bias, and religious bias. |
| Approach: | They propose an algorithm that uses a neural network to perform ‘soft debiasing’ and build on the seminal work of (CITATION) and (CitATION). |
| Outcome: | The proposed algorithm outperforms current methods on gender, race, and religion metrics on a wide range of metrics. |
Re-examining Sexism and Misogyny Classification with Annotator Attitudes (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing datasets for content moderation fail to capture plurality of possible annotator perspectives or ensure representation of affected groups. |
| Approach: | They examine the relationship between annotator identities and attitudes and the responses they give to two GBV labelling tasks. |
| Outcome: | The results show that higher Right Wing Authoritarianism scores are associated with a higher propensity to label text as sexist . higher scores are also associated with negative attitudes towards sexism and neosexist attitudes . |
Towards Robustifying NLI Models Against Lexical Dataset Biases (2020.acl-main)
Copied to clipboard
| Challenge: | Recent studies show that deep learning models exploit dataset biases without deep understanding of the language semantics. |
| Approach: | They propose two methods to debiase models against lexical dataset biases . they use contradiction-word bias and word-overlapping bias as examples . |
| Outcome: | The proposed method removes label bias at embedding level, while the other uses a bag-of-words sub-model to capture features likely to exploit the bias. |
Towards Building More Robust NER datasets: An Empirical Study on NER Dataset Bias from a Dataset Difficulty View (2023.emnlp-main)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) models rely on superficial entity patterns for predictions, without considering evidence from the context. |
| Approach: | They propose to de-bias NER datasets by altering entity-context distribution . they also validate the feasibility of the proposed de-bianking techniques . |
| Outcome: | The proposed methods can be applied to different models and improve existing models. |
Fighting Bias With Bias: Promoting Model Robustness by Amplifying Dataset Biases (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent work sought to develop robust, unbiased models by filtering biased examples from training sets. |
| Approach: | They propose to filter out biased examples from training sets to improve models' performance. |
| Outcome: | The proposed evaluation framework is more challenging than the original dataset splits and even more challenging that hand-crafted challenge sets. |