| Challenge: | Abusive language detection models tend to be biased toward identity words of a certain group of people . recent studies have raised concerns about the robustness of such systems . |
| Approach: | They propose to use debiased word embeddings, gender swap data augmentation to reduce model bias . they also propose to fine-tune models with a larger corpus to correct such bias if needed . |
| Outcome: | The proposed methods reduce model bias by 90-98% and can be extended to correct model bias in other scenarios. |
Similar Papers
Detection of Abusive Language: the Problem of Biased Datasets (N19-1)
Copied to clipboard
| Challenge: | Recent studies have reported high classification performance on datasets with difficult cases of abusive language. |
| Approach: | They examine the impact of data bias on abusive language detection by focusing on specific microposts rather than random sampling. |
| Outcome: | The proposed method is more accurate and more accurate than random sampling. |
Identifying and Reducing Gender Bias in Word-Level Language Models (N19-3)
Copied to clipboard
| Challenge: | Existing discriminatory biases in training data can be amplified by models . text corpora exhibit socially problematic biase . |
| Approach: | They propose a metric to measure gender bias and a regularization loss term to minimize embeddings onto an embeddable subspace that encodes gender. |
| Outcome: | The proposed method reduces gender bias up to an optimal weight assigned to the loss term, and the model becomes unstable as the perplexity increases. |
Language Models Get a Gender Makeover: Mitigating Gender Bias with Few-Shot Data Interventions (2023.acl-short)
Copied to clipboard
| Challenge: | Existing approaches to de-bias pre-trained large language models focus on changes to training regime, but this is not feasible. |
| Approach: | They propose to de-bias a pre-trained model by fine-tuning it on only 10 examples . they show that the technique performs better than competitive baselines . |
| Outcome: | The proposed method performs better than competitive state-of-the-art baselines with minimal loss in language modeling ability. |
Mitigating Gender Bias in Natural Language Processing: Literature Review (P19-1)
Copied to clipboard
Tony Sun, Andrew Gaut, Shirlyn Tang, Yuxin Huang, Mai ElSherief, Jieyu Zhao, Diba Mirza, Elizabeth Belding, Kai-Wei Chang, William Yang Wang
| Challenge: | NLP models propagate and may even amplify gender bias found in text corpora . methods to mitigate gender bias in NLP are relatively nascent . |
| Approach: | They propose to analyze gender bias based on four forms of representation bias and discuss the advantages and drawbacks of existing gender debiasing methods. |
| Outcome: | The proposed methods are based on four forms of representation bias and have advantages and drawbacks. |
Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Them (N19-1)
Copied to clipboard
| Challenge: | Existing methods to remove gender bias from word embeddings are insufficient, we argue . existing methods for gender-neutral modeling are ineffective, we conclude . |
| Approach: | They propose methods to reduce gender bias in word embeddings by debiasing them using text corpora. |
| Outcome: | The proposed methods show that they can reduce gender bias in word embeddings . the proposed methods are insufficient and should not be trusted, the authors argue . |
Projective Methods for Mitigating Gender Bias in Pre-trained Language Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Mitigating gender bias in NLP has a long history tied to debiasing static word embeddings. |
| Approach: | They propose a masked language modelling task where content is developed around known social stereotypes and a projective debiasing method is used to reduce bias. |
| Outcome: | The proposed methods reduce intrinsic bias and mitigat observed bias in a downstream setting, but the two outcomes are not necessarily correlated. |
Reducing Gender Bias in Word-Level Language Models with a Gender-Equalizing Loss Function (P19-2)
Copied to clipboard
| Challenge: | Existing methods to reduce gender bias in natural language datasets are inadequate. |
| Approach: | They propose a loss function modification approach which equalizes the probabilities of male and female words in the output. |
| Outcome: | The proposed approach outperforms existing methods in several aspects, especially in reducing gender bias in occupation words. |
Examining Gender Bias in Languages with Grammatical Gender (D19-1)
Copied to clipboard
| Challenge: | Existing studies on gender bias in word embeddings focus on English . however, these studies cannot be extended to languages with morphological agreement on gender . |
| Approach: | They propose new metrics to evaluate gender bias in word embeddings of English and Spanish . they extend existing approaches to mitigate gender bias while preserving original embeddables . |
| Outcome: | The proposed methods reduce gender bias while preserving the original embeddings. |
Black is to Criminal as Caucasian is to Police: Detecting and Removing Multiclass Bias in Word Embeddings (N19-1)
Copied to clipboard
| Challenge: | Existing methods to debias word embeddings in binary settings such as gender and religion are limited to binary labels, whereas word2vec embedders can be used to propagate biases. |
| Approach: | They propose a method to debias word embeddings in multiclass settings such as gender and religion, extending the work of Bolukbasi et al. (2016). |
| Outcome: | The proposed method maintains the efficacy in standard NLP tasks while maintaining the utility of embeddings. |
Leveraging Pre-trained Language Models for Gender Debiasing (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing methods to reduce gender bias in natural language are costly and time-consuming. |
| Approach: | They propose a method to generate gender variants for a given text using pre-trained language models as the resource without any task-specific labelled data. |
| Outcome: | The proposed method can reduce gender bias in a language generation context without a task-specific labelled data. |