Unlabeled Debiasing in Downstream Tasks via Class-wise Low Variance Regularization (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for debiasing depend on attribute labels and target attributes. |
| Approach: | They propose a method that uses class-wise variance of embeddings to reduce the effects of debiasing on a downstream task. |
| Outcome: | The proposed method outperforms baselines that rely on attribute labels while maintaining performance on the target task. |
Similar Papers
Applying Intrinsic Debiasing on Downstream Tasks: Challenges and Considerations for Machine Translation (2024.emnlp-main)
Copied to clipboard
| Challenge: | In this study, we examine three considerations for intrinsic debiasing in neural machine translation models. |
| Approach: | They propose to measure the extrinsic bias of neural machine translation models by embedding them in a neural embeddable space and using different tokens to debias them. |
| Outcome: | The proposed methods over-rely on gender stereotypes and over-represent them in their models. |
Gender-tuning: Empowering Fine-tuning for Debiasing Pre-trained Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for debiasing are resource-intensive and costly. Existing solutions for debiansing require fine-tuning on downstream tasks. |
| Approach: | They propose to integrate Masked Language Modeling (MLM) training objectives into fine-tuning’s training process to debiase the PLMs. |
| Outcome: | The proposed approach outperforms the state-of-the-art baselines in terms of gender bias scores while improving PLMs’ performance solely using the downstream tasks’ dataset. |
Causal-Debias: Unifying Debiasing in Pretrained Language Models and Fine-tuning via Causal Invariant Learning (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods to remove unwanted stereotypical associations from pretrained language models (PLMs) are often focused on removing unwanted stereotypes from PLMs. |
| Approach: | They propose a framework to remove unwanted stereotypical associations in pretrained language models . they propose bias-relevant factors are causal, while labelrelevant factors causal . |
| Outcome: | The proposed framework reduces stereotypical associations after PLMs are fine-tuned . the proposed framework mitigates bias from a causal invariant perspective . |
Debiasing Pre-trained Contextualised Embeddings (2021.eacl-main)
Copied to clipboard
| Challenge: | a study of contextualised word embeddings shows discriminative biases are encoded in contextualised embeddables. |
| Approach: | They propose a fine-tuning method that can be applied at token- or sentence-levels to debias pre-trained contextualised embeddings. |
| Outcome: | The proposed method can be applied at token- or sentence-levels to debias pre-trained models without requiring retrains. |
Gender-preserving Debiasing for Pre-trained Word Embeddings (P19-1)
Copied to clipboard
| Challenge: | Existing methods for debiasing word embeddings have shown discriminative biases . word embeds learnt from social media have shown to encode racist, offensive and discriminative language usage. |
| Approach: | They propose a method that preserves gender-related information while removing stereotypical gender biases from pre-trained word embeddings. |
| Outcome: | The proposed method preserves gender-related information while removing stereotypical discriminative gender biases from pre-trained word embeddings. |
Debiasing knowledge graph embeddings (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to train knowledge graph embeddings to be neutral to sensitive attributes such as gender have been shown to increase training time by a factor of eight or more. |
| Approach: | They propose a method where all embeddings are trained to be neutral to sensitive attributes such as gender by default using an adversarial loss. |
| Outcome: | The proposed method reduces training time by eightfold and improves accuracy. |
Identifying and Reducing Gender Bias in Word-Level Language Models (N19-3)
Copied to clipboard
| Challenge: | Existing discriminatory biases in training data can be amplified by models . text corpora exhibit socially problematic biase . |
| Approach: | They propose a metric to measure gender bias and a regularization loss term to minimize embeddings onto an embeddable subspace that encodes gender. |
| Outcome: | The proposed method reduces gender bias up to an optimal weight assigned to the loss term, and the model becomes unstable as the perplexity increases. |
Marked Attribute Bias in Natural Language Inference (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing tests for gender-biased word embeddings do not address marked attribute bias . authors propose a new type of intrinsic bias measure for static word embeds . |
| Approach: | They propose a method to detect gender-biased word embeddings in a downstream NLP application . they propose 'debiasing' method to measure the marked attribute bias in embeddable word embeds . |
| Outcome: | The proposed method achieves best results on the marked attribute bias test set. |
Modular and On-demand Bias Mitigation with Attribute-Removal Subnetworks (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing studies show that pre-trained language models can be used to mitigate societal biases and stereotypes. |
| Approach: | They propose a modular bias mitigation approach that integrates debiasing modules into the core model on-demand at inference time. |
| Outcome: | The proposed approach improves on-par with baseline finetuning on gender, race, and age protected attributes on three classification tasks with gender, age, and race as protected attributes. |
Neutralizing Gender Bias in Word Embeddings with Latent Disentanglement and Counterfactual Generation (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Recent research shows word embeddings have strong gender biases in embeddable spaces . a proposed method can be used to debiase word embeds without loss of semantic information . |
| Approach: | They propose a latent disentanglement method with a siamese auto-encoder structure with an adapted gradient reversal layer to debiase word embeddings. |
| Outcome: | The proposed method can preserve semantic information during debiasing while minimizing loss of semantic information for extrinsic NLP tasks. |