Debiasing Isn’t Enough! – on the Effectiveness of Debiasing MLMs and Their Social Biases in Downstream Tasks (2022.coling-1)
Copied to clipboard
| Challenge: | Existing measures for social bias evaluation are inadequate for MLMs to accurately evaluate the social biases in their systems. |
| Approach: | They propose task-agnostic intrinsic and task-specific extrinsic social bias evaluation measures for MLMs that use different methods to re-learn social biases during fine-tuning on downstream tasks. |
| Outcome: | The findings highlight the limitations of existing MLM bias evaluation measures and raise concerns on the deployment of MLMs in downstream applications using those measures. |
Similar Papers
Applying Intrinsic Debiasing on Downstream Tasks: Challenges and Considerations for Machine Translation (2024.emnlp-main)
Copied to clipboard
| Challenge: | In this study, we examine three considerations for intrinsic debiasing in neural machine translation models. |
| Approach: | They propose to measure the extrinsic bias of neural machine translation models by embedding them in a neural embeddable space and using different tokens to debias them. |
| Outcome: | The proposed methods over-rely on gender stereotypes and over-represent them in their models. |
Towards Understanding Task-agnostic Debiasing Through the Lenses of Intrinsic Bias and Forgetfulness (2024.findings-acl)
Copied to clipboard
Guangliang Liu, Milad Afshari, Xitong Zhang, Zhiyu Xue, Avrajit Ghosh, Bidhan Bashyal, Rongrong Wang, Kristen Johnson
| Challenge: | Debiasing Pretrained Language Models (PLMs) are task-agnostic and can be generalizable, but its impact on language modeling ability and the risk of relearning social biases remain as the two most significant challenges. |
| Approach: | They propose a framework which can Propagate Socially-fair Debiasing to Downstream Fine-tuning to alleviate the forgetting issue of PLMs by regularizing debiased attention heads based on the PLM’s bias levels from stages of pretraining and debiase. |
| Outcome: | The proposed framework can Propagate Socially-fair Debiasing to Downstream Fine-tuning, indicating that the ineffectiveness of debiase can be alleviated by overcoming the forgetting issue through regularizing successfully debiased attention heads based on the PLMs’ bias levels from stages of pretraining and debiases. |
The Gaps between Fine Tuning and In-context Learning in Bias Evaluation and Debiasing (2025.coling-main)
Copied to clipboard
| Challenge: | FT-based debiasing methods cause a performance degradation in downstream tasks . FT works by updating some or all parameters, while ICL uses prompts without modifying the model parameters. |
| Approach: | They propose to use ICL to customize PLMs to downstream tasks without parameter updates. |
| Outcome: | The proposed method lowers the performance degradation of FT-based debiasing methods compared to FT models . the proposed method improves performance on large datasets while allowing for smaller changes to PLMs . |
How Gender Debiasing Affects Internal Model Representations, and Why It Matters (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing studies of gender bias in NLP focus on extrinsic or intrinsic bias, but the relationship between extrindic and intrinsic bias is relatively unknown. |
| Approach: | They propose a framework to measure extrinsic and intrinsic bias together and propose metric to measure debiasing and intrinsic debiases. |
| Outcome: | The proposed framework provides a comprehensive perspective on bias in NLP models, which can be applied to deploy NLP systems in a more informed manner. |
Intrinsic Bias Metrics Do Not Correlate with Application Bias (2021.acl-long)
Copied to clipboard
| Challenge: | a recent survey of bias in natural language processing found that a coreference system makes more errors in an anti-stereotypical coreferent than in a pro-sterereotype one. |
| Approach: | They compare intrinsic and extrinsic bias metrics across hundreds of trained models . they urge researchers to focus on extrindic measures of bias, not easy to measure . |
| Outcome: | a new intrinsic metric and an annotated test set on gender bias in hate speech are tested . authors urge researchers to focus on extrinsic measures of bias, and to make them more feasible . |
A Predictive Factor Analysis of Social Biases and Task-Performance in Pretrained Masked Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Various types of social biases have been reported with pretrained Masked Language Models (MLMs) in prior work. |
| Approach: | They conduct a comprehensive study on 39 pretrained MLMs to examine their model factors and their social biases. |
| Outcome: | The proposed model factors influence social biases learned by an MLM and their downstream task performance. |
Robust Bias Detection in MLMs and its Application to Human Trait Ratings (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods to assess demographic bias in MLMs ignore random variability of templates and target concepts, and neglect bias quantification. |
| Approach: | They propose a systematic statistical approach to assess bias in MLMs using mixed models to account for random effects, pseudo-perplexity weights for sentences derived from templates and quantify bias using statistical effect sizes. |
| Outcome: | The proposed method matches on bias scores in magnitude and direction with small to medium effect sizes. |
Quantifying Social Biases in NLP: A Generalization and Empirical Comparison of Extrinsic Fairness Metrics (2021.tacl-1)
Copied to clipboard
| Challenge: | Existing fairness metrics quantify the differences in a model’s behaviour across a range of demographic groups. |
| Approach: | They propose to unify existing fairness metrics and compare them to three generalized fairness measures to reveal the connections between them. |
| Outcome: | The proposed measures can be explained by differences in parameter choices, and the results are consistent with previous studies. |
When Do Pre-Training Biases Propagate to Downstream Tasks? A Case Study in Text Summarization (2023.eacl-main)
Copied to clipboard
Faisal Ladhak, Esin Durmus, Mirac Suzgun, Tianyi Zhang, Dan Jurafsky, Kathleen McKeown, Tatsunori Hashimoto
| Challenge: | Existing studies have shown that large language models contain linguistic and societal biases, but it is unclear how these biase amplify to downstream tasks. |
| Approach: | They investigate how name-nationality bias propagates from pre-training to downstream tasks . they show that these biases manifest themselves as hallucinations in summarization . |
| Outcome: | The proposed model can reduce the rate of hallucinations, but does not change the types of biases that do appear. |
Analyzing Effects of Learning Downstream Tasks on Moral Bias in Large Language Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for fine-tuning large language models replicate and perpetuate social biases . pre-existing moral bias may be mitigated or amplified even when presented with opposing views . |
| Approach: | They develop methods to assess the agreement of LMs to explicit codified norms . they find that introducing downstream tasks may lead to unexpected inconsistencies . |
| Outcome: | The proposed model can be used to improve morality in data-scarce tasks. |