Comparative Evaluation of Label-Agnostic Selection Bias in Multilingual Hate Speech Datasets (2020.emnlp-main)
Copied to clipboard
| Challenge: | a recent study has shown that data collection is neglected by ignoring the quality of data. |
| Approach: | They propose to use latent semantics to evaluate selection bias in hate speech . they compare latent Dirichlet Allocation (LDA) to eleven hate speech corpora . |
| Outcome: | The proposed method could be revisable before focusing on classification performance. |
Similar Papers
Latent Hatred: A Benchmark for Understanding Implicit Hate Speech (2021.emnlp-main)
Copied to clipboard
Mai ElSherief, Caleb Ziems, David Muchlinski, Vaishnavi Anupindi, Jordyn Seybolt, Munmun De Choudhury, Diyi Yang
| Challenge: | Existing studies on explicit or overt hate speech have failed to address a more pervasive form based on coded or indirect language. |
| Approach: | They propose a theoretically-justified taxonomy of implicit hate speech and a benchmark corpus with fine-grained labels for each message and its implication. |
| Outcome: | The proposed dataset will serve as a useful benchmark for understanding this multifaceted issue. |
Exploring Cross-Cultural Differences in English Hate Speech Annotations: From Dataset Construction to Analysis (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing datasets for hate speech detection neglect the cultural diversity within a single language. |
| Approach: | They propose a CR**oss-cultural **E**nglish **Hate* speech dataset that uses culturally hateful keywords to identify posts from four countries plus the United States. |
| Outcome: | The proposed dataset shows that only 56.2% of the posts in CREHate achieve consensus among all countries, with the highest pairwise label difference rate of 26%. |
Toxic, Hateful, Offensive or Abusive? What Are We Really Classifying? An Empirical Analysis of Hate Speech Datasets (2020.lrec-1)
Copied to clipboard
| Challenge: | a recent study shows that many definitions are being used for equivalent concepts, making most datasets incompatible. |
| Approach: | They analyze six publicly available datasets to determine their similarity and compatibility . they propose to use Fast Text word vectors to analyze similarity between different datasets . |
| Outcome: | The proposed model performs better on similar datasets and worse on more non-offensive samples. |
Multilingual Twitter Corpus and Baselines for Evaluating Demographic Bias in Hate Speech Recognition (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing work on document classification models mainly uses synthetic monolingual data without ground truth for author demographic attributes. |
| Approach: | They assemble and publish a multilingual Twitter corpus for the task of hate speech detection using inferred author demographic factors. |
| Outcome: | The results show that the classifiers learn human biases and can be discriminatory towards certain demographic groups. |
Multilingual and Multi-Aspect Hate Speech Analysis (D19-1)
Copied to clipboard
| Challenge: | Current research on hate speech analysis is oriented towards monolingual and single classification tasks. |
| Approach: | They propose to use a multilingual multi-aspect hate speech analysis dataset to test current methods . they evaluate the dataset in various classification settings and discuss how to leverage annotations . |
| Outcome: | The proposed dataset can be used to improve hate speech detection and classification in general. |
The Risk of Racial Bias in Hate Speech Detection (P19-1)
Copied to clipboard
| Challenge: | Annotators’ insensitivity to differences in dialect can lead to racial bias in automatic hate speech detection models, potentially amplifying harm against minority populations. |
| Approach: | They propose *dialect* and *race priming* as ways to reduce the racial bias in hate speech detection models by detecting differences in dialects in annotated tweets. |
| Outcome: | The proposed models acquire and propagate these biases, such that AAE tweets and tweets by self-identified African Americans are up to two times more likely to be labelled as offensive compared to others. |
The Challenges of Creating a Parallel Multilingual Hate Speech Corpus: An Exploration (2024.lrec-main)
Copied to clipboard
| Challenge: | Hate speech is one of the most demanding topics in Natural Language Processing, as its multifaceted nature is accompanied by a handful of challenges, such as multilinguality and cross-linguality. |
| Approach: | They propose a pipeline that could be used to create a parallel multilingual hate speech dataset using machine translation. |
| Outcome: | The proposed pipeline will be able to create a parallel multilingual hate speech dataset using machine translation. |
What the #?*!: Disentangling Hate Across Target Identities (2025.naacl-long)
Copied to clipboard
| Challenge: | Hate speech classifiers do not perform equally well in detecting hateful expressions towards different target identities. |
| Approach: | They propose to use two recently proposed functionality test datasets to analyze the impact of different factors on HS prediction. |
| Outcome: | The proposed classifiers do not perform equally well across different datasets and different target identities. |
HateDay: Insights from a Global Hate Speech Dataset Representative of a Day on Twitter (2025.acl-long)
Copied to clipboard
Manuel Tonneau, Diyi Liu, Niyati Malhotra, Scott A. Hale, Samuel Fraiberger, Victor Orozco-Olvera, Paul Röttger
| Challenge: | Prior work on automated hate speech detection models has been limited due to systematic biases in evaluation datasets and poor performance across geographies. |
| Approach: | They propose to construct a global hate speech dataset representative of social media settings from tweets posted on September 21, 2022. |
| Outcome: | The proposed dataset covers eight languages and four English-speaking countries and covers eight countries where English is the main language on Twitter. |
Social Bias in Multilingual Language Models: A Survey (2025.emnlp-main)
Copied to clipboard
| Challenge: | Pretrained multilingual models exhibit the same social bias as models processing English texts. |
| Approach: | They examine the literature on bias evaluation and mitigation approaches in multilingual and non-English contexts and identify gaps in the field. |
| Outcome: | The proposed models perform well on multilingual language understanding benchmarks and are consistent with the current literature. |