Improving Gender Fairness of Pre-Trained Language Models without Catastrophic Forgetting (2023.acl-short)
Copied to clipboard
| Challenge: | Existing studies addressing gender bias of pre-trained language models, usually build a small gender-neutral data set and conduct a second phase pre-training with such data. |
| Approach: | They propose a method to improve gender fairness of pre-trained models with less forgetting by evaluating them with general NLP tasks in GLUE. |
| Outcome: | The proposed method improves gender fairness of pre-trained models with less forgetting and performs better on GLUE by a large margin. |
Similar Papers
Language Models Get a Gender Makeover: Mitigating Gender Bias with Few-Shot Data Interventions (2023.acl-short)
Copied to clipboard
| Challenge: | Existing approaches to de-bias pre-trained large language models focus on changes to training regime, but this is not feasible. |
| Approach: | They propose to de-bias a pre-trained model by fine-tuning it on only 10 examples . they show that the technique performs better than competitive baselines . |
| Outcome: | The proposed method performs better than competitive state-of-the-art baselines with minimal loss in language modeling ability. |
Evaluating Bias and Fairness in Gender-Neutral Pretrained Vision-and-Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Pretrained machine learning models perpetuate and even amplify existing biases in data . this can result in unfair outcomes that ultimately impact user experience . |
| Approach: | They quantify bias amplification in pretraining and after fine-tuning on vision-and-language models. |
| Outcome: | The results show that pretrained models can perpetuate and even amplify biases in data without compromising performance. |
In-Contextual Gender Bias Suppression for Large Language Models (2024.findings-eacl)
Copied to clipboard
| Challenge: | Prior work has proposed debiasing methods that require human labelled examples, data augmentation and fine-tuning of LLMs, which are computationally expensive. |
| Approach: | They propose to suppress gender biases by providing textual preambles from manually designed templates and real-world statistics without accessing model parameters. |
| Outcome: | The proposed methods suppress gender biases in English LLMs using a CrowsPairs dataset without accessing model parameters. |
An Empirical Analysis of Parameter-Efficient Methods for Debiasing Pre-Trained Language Models (2023.acl-long)
Copied to clipboard
| Challenge: | Pre-trained language models inherit more human-like biases from the training corpora, causing computationally expensive problems. |
| Approach: | They propose parameter-efficient methods in combination with counterfactual data augmentation for bias mitigation. |
| Outcome: | The proposed methods are effective in mitigating gender bias, prompt tuning is more suitable for GPT-2 than BERT, and less effective when it comes to racial and religious bias. |
Reducing Gender Bias in Neural Machine Translation as a Domain Adaptation Problem (2020.acl-main)
Copied to clipboard
| Challenge: | Training data for NLP tasks often exhibits gender bias in that fewer sentences refer to women than to men. |
| Approach: | They propose a lattice-rescoring scheme which allows a trade-off between general translation quality and bias reduction during adaptation and inference time. |
| Outcome: | The proposed approach outperforms all systems evaluated on WinoMT with no degradation of general test set BLEU. |
Leveraging Pre-trained Language Models for Gender Debiasing (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing methods to reduce gender bias in natural language are costly and time-consuming. |
| Approach: | They propose a method to generate gender variants for a given text using pre-trained language models as the resource without any task-specific labelled data. |
| Outcome: | The proposed method can reduce gender bias in a language generation context without a task-specific labelled data. |
Collecting a Large-Scale Gender Bias Dataset for Coreference Resolution and Machine Translation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies have found evidence of gender bias in machine translation and coreference resolution models using mostly synthetic diagnostic datasets. |
| Approach: | They propose a semi-automatic method to vastly extend synthetic, small diagnostic datasets to include grammatical patterns indicating stereotypical and non-stereotypical gender-role assignments. |
| Outcome: | The proposed method extends the existing dataset to 108K diverse English sentences. |
Gender-preserving Debiasing for Pre-trained Word Embeddings (P19-1)
Copied to clipboard
| Challenge: | Existing methods for debiasing word embeddings have shown discriminative biases . word embeds learnt from social media have shown to encode racist, offensive and discriminative language usage. |
| Approach: | They propose a method that preserves gender-related information while removing stereotypical gender biases from pre-trained word embeddings. |
| Outcome: | The proposed method preserves gender-related information while removing stereotypical discriminative gender biases from pre-trained word embeddings. |
Mitigating Gender Bias in Natural Language Processing: Literature Review (P19-1)
Copied to clipboard
Tony Sun, Andrew Gaut, Shirlyn Tang, Yuxin Huang, Mai ElSherief, Jieyu Zhao, Diba Mirza, Elizabeth Belding, Kai-Wei Chang, William Yang Wang
| Challenge: | NLP models propagate and may even amplify gender bias found in text corpora . methods to mitigate gender bias in NLP are relatively nascent . |
| Approach: | They propose to analyze gender bias based on four forms of representation bias and discuss the advantages and drawbacks of existing gender debiasing methods. |
| Outcome: | The proposed methods are based on four forms of representation bias and have advantages and drawbacks. |
Exploiting Biased Models to De-bias Text: A Gender-Fair Rewriting Model (2023.acl-long)
Copied to clipboard
| Challenge: | Existing work has explored using sequence-to-sequence rewriting models to transform biased outputs into more gender-fair language by creating pseudo training data through linguistic rules. |
| Approach: | They propose to use machine translation models to create gender-biased text from real gender-fair text via round-trip translation to eliminate rule-based data creation. |
| Outcome: | The proposed approach matches the performance of state-of-the-art rewriting models for English. |