Multilingual Twitter Corpus and Baselines for Evaluating Demographic Bias in Hate Speech Recognition (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing work on document classification models mainly uses synthetic monolingual data without ground truth for author demographic attributes. |
| Approach: | They assemble and publish a multilingual Twitter corpus for the task of hate speech detection using inferred author demographic factors. |
| Outcome: | The results show that the classifiers learn human biases and can be discriminatory towards certain demographic groups. |
Similar Papers
Social Bias in Multilingual Language Models: A Survey (2025.emnlp-main)
Copied to clipboard
| Challenge: | Pretrained multilingual models exhibit the same social bias as models processing English texts. |
| Approach: | They examine the literature on bias evaluation and mitigation approaches in multilingual and non-English contexts and identify gaps in the field. |
| Outcome: | The proposed models perform well on multilingual language understanding benchmarks and are consistent with the current literature. |
An Italian Twitter Corpus of Hate Speech against Immigrants (L18-1)
Copied to clipboard
| Challenge: | a recent study has annotated 6,000 tweets for hate speech against immigrants . the annotation scheme was designed to account for the multiplicity of factors that can contribute to the definition of a hate speech notion . |
| Approach: | They describe a Twitter corpus annotated for hate speech against immigrants . they propose a scheme that includes aggressiveness, offensiveness, irony, stereotype and intensity . |
| Outcome: | The proposed annotation scheme includes aggressiveness, offensiveness, irony, stereotype, intensity and (on an experimental basis) intensity. |
Easy Adaptation to Mitigate Gender Bias in Multilingual Text Classification (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to mitigate demographic biases evaluate on monolingual data, however, multilingual data has not been examined. |
| Approach: | They propose a standard domain adaptation model to reduce gender bias in multilingual contexts. |
| Outcome: | The proposed model reduces gender bias and improves on two text classification tasks with three fair-aware baselines. |
Comparative Evaluation of Label-Agnostic Selection Bias in Multilingual Hate Speech Datasets (2020.emnlp-main)
Copied to clipboard
| Challenge: | a recent study has shown that data collection is neglected by ignoring the quality of data. |
| Approach: | They propose to use latent semantics to evaluate selection bias in hate speech . they compare latent Dirichlet Allocation (LDA) to eleven hate speech corpora . |
| Outcome: | The proposed method could be revisable before focusing on classification performance. |
The Challenges of Creating a Parallel Multilingual Hate Speech Corpus: An Exploration (2024.lrec-main)
Copied to clipboard
| Challenge: | Hate speech is one of the most demanding topics in Natural Language Processing, as its multifaceted nature is accompanied by a handful of challenges, such as multilinguality and cross-linguality. |
| Approach: | They propose a pipeline that could be used to create a parallel multilingual hate speech dataset using machine translation. |
| Outcome: | The proposed pipeline will be able to create a parallel multilingual hate speech dataset using machine translation. |
Bias Beyond English: Counterfactual Tests for Bias in Sentiment Analysis in Four Languages (2023.findings-acl)
Copied to clipboard
| Challenge: | Sentiment analysis systems are used in hundreds of products and languages . Gender and racial biases are well-studied in English, but understudied elsewhere . |
| Approach: | They build a counterfactual evaluation corpus for gender and racial/migrant bias in four languages. |
| Outcome: | The evaluation corpus reveals which models have less bias and pinpoints changes in model bias behaviour, enabling more targeted mitigation strategies. |
Comparing Biases and the Impact of Multilingual Training across Multiple Languages (2023.emnlp-main)
Copied to clipboard
Sharon Levy, Neha John, Ling Liu, Yogarshi Vyas, Jie Ma, Yoshinari Fujinuma, Miguel Ballesteros, Vittorio Castelli, Dan Roth
| Challenge: | Currently, studies on bias and fairness in natural language processing focus on a single language and/or across few attributes (e.g. gender, race). However, biases can manifest differently across languages for individual attributes. |
| Approach: | They adapt existing sentiment bias templates in English to Italian, Chinese, Hebrew, and Spanish for race, religion, nationality, and gender. |
| Outcome: | The proposed model favors groups that are dominant in each language's culture, indicating bias amplification, after multilingual finetuning. |
On Evaluating and Mitigating Gender Biases in Multilingual Settings (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks and resources for evaluating gender biases in multilingual settings are limited. |
| Approach: | They propose to extend DisCo to different Indian languages using human annotations to evaluate gender biases in multilingual models. |
| Outcome: | The proposed benchmarks and mitigation techniques are extended beyond English to evaluate gender biases in multilingual models. |
Multilingual and Multi-Aspect Hate Speech Analysis (D19-1)
Copied to clipboard
| Challenge: | Current research on hate speech analysis is oriented towards monolingual and single classification tasks. |
| Approach: | They propose to use a multilingual multi-aspect hate speech analysis dataset to test current methods . they evaluate the dataset in various classification settings and discuss how to leverage annotations . |
| Outcome: | The proposed dataset can be used to improve hate speech detection and classification in general. |
Artie Bias Corpus: An Open Dataset for Detecting Demographic Bias in Speech Applications (2020.lrec-1)
Copied to clipboard
| Challenge: | A speech technology exhibits demographic bias when performance is worse for one demographic group relative to another. |
| Approach: | They create an English dataset of expert-validated audio, transcript> pairs with demographic tags for age, gender, accent and open software which may be used to detect demographic bias in Automatic Speech Recognition systems. |
| Outcome: | The Artie Bias Corpus is a curated subset of the Mozilla Common Voice corpus, which is released under a Creative Commons CC0 license . |