| Challenge: | Existing methods to reduce biases in pre-training models are hampered by their performance. |
| Approach: | They propose a method that utilizes structured knowledge to mitigate bias in LLMs . their method obviates the need for training from scratch, thus offering enhanced scalability . |
| Outcome: | The proposed method outperforms state-of-the-art (SOTA) baselines in the debiasing ability. |
Similar Papers
Self-Supervised Position Debiasing for Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for debiasing large language models require external bias knowledge or annotated non-biased samples, which is lacking for position debiases. |
| Approach: | They propose a self-supervised position debiasing framework that leverages unsupervised responses from pre-trained LLMs for debiazing without external bias knowledge. |
| Outcome: | The proposed framework outperforms existing methods in mitigating three types of position biases on eight datasets and five tasks. |
From Prejudice to Parity: A New Approach to Debiasing Large Language Model Word Embeddings (2025.coling-main)
Copied to clipboard
| Challenge: | Existing work in this field has looked most commonly into gender bias, racial bias, and religious bias. |
| Approach: | They propose an algorithm that uses a neural network to perform ‘soft debiasing’ and build on the seminal work of (CITATION) and (CitATION). |
| Outcome: | The proposed algorithm outperforms current methods on gender, race, and religion metrics on a wide range of metrics. |
Understanding the Effect of Model Compression on Social Bias in Large Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) trained with self-supervision on vast corpora of web text fit to the social biases of that text, leading to representational harm. |
| Approach: | They propose to use quantization and knowledge distillation to reduce the computational burden of LLMs to mitigate the effects of inappropriate social biases learned during pretraining. |
| Outcome: | The proposed methods reduce the computational burden of large language models by reducing their size and complexity. |
End-to-End Bias Mitigation by Modelling Biases in Corpora (2020.acl-main)
Copied to clipboard
| Challenge: | Recent studies have shown that strong natural language understanding models are prone to relying on unwanted dataset biases without learning the underlying task. |
| Approach: | They propose two learning strategies to train neural models that are more robust to dataset biases and transfer better to out-of-domain datasets. |
| Outcome: | The proposed methods improve robustness in all settings and transfer better to out-of-domain datasets. |
Large Language Models Are Still Misled by Simple Bias Ensembles (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks for large language models are constrained to datasets where each sample is manually injected with only one type of bias. |
| Approach: | They propose a multi-bias benchmark where each sample contains multiple types of biases. |
| Outcome: | The proposed benchmark shows that existing LLMs and debiasing methods perform poorly on this benchmark, highlighting the challenge of eliminating compounded biases. |
Identifying and Mitigating Social Bias Knowledge in Language Models (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for debiasing may generate incorrect or nonsensical predictions but leave aside individual commonsense facts, resulting in modified knowledge that elicits unreasonable or undesired predictions. |
| Approach: | They propose a framework that identifies encoding locations of biases within language models and then applies the Fairness-Stamp (FAST) they also propose 'BiaScope' to evaluate the retention of commonsense knowledge and generalization across paraphrased social biase. |
| Outcome: | The proposed framework surpasses state-of-the-art baselines with superior debiasing performance while not compromising the overall model capability for knowledge retention and prediction. |
Data-Centric Explainable Debiasing for Improving Fairness in Pre-trained Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing data-centric debiasing strategies mainly leverage explicit bias words for counterfactual data augmentation to balance the training data. |
| Approach: | They propose a method which uses an explainability method to search for implicit bias words to assist in debiasing PLMs. |
| Outcome: | Extensive results show that the proposed method achieves state-of-the-art debiasing performance and strong generalization while maintaining predictive abilities. |
Conceptor-Aided Debiasing of Large Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained large language models reflect inherent social biases of their training corpus. |
| Approach: | They propose two methods to identify and remove the bias subspace in pre-trained large language models such as BERT and GPT by applying conceptors to a conceptor NOT operation. |
| Outcome: | The proposed method achieves state-of-the-art (SoTA) debiasing while maintaining LLMs’ performance on the GLUE benchmark. |
BiasFilter: An Inference-Time Debiasing Framework for Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for debiasing large language models incur high human and computational costs and are limited in their effectiveness. |
| Approach: | They propose a model-agnostic, inference-time debiasing framework that enforces fairness by filtering generation outputs in real time. |
| Outcome: | The proposed framework mitigates social bias across a range of LLMs while preserving overall generation quality. |
MAFIA: Multi-Adapter Fused Inclusive Language Models (2024.eacl-long)
Copied to clipboard
| Challenge: | Pretrained Language Models (PLMs) are widely used in NLP for various tasks. |
| Approach: | They propose to modularly debias a pre-trained language model across multiple bias dimensions using structured knowledge and a large generative model. |
| Outcome: | The proposed model is able to debias a pre-trained language model across multiple bias dimensions in a semi-automated way. |