Bias after Prompting: Persistent Discrimination in Large Language Models (2025.findings-emnlp)
Copied to clipboard
Nivedha Sivakumar, Natalie Mackraz, Samira Khorshidi, Krishna Patel, Barry-John Theobald, Luca Zappella, Nicholas Apostoloff
| Challenge: | a dangerous assumption is that biases do not transfer from pre-trained large language models to adapted models. |
| Approach: | They validate the bias transfer hypothesis by using prompt adaptations to study biases in causal models . they find that popular prompt-based mitigation methods do not consistently prevent biase transferring . |
| Outcome: | The results invalidate the assumption that biases do not transfer from pre-trained models to adapted models. |
Similar Papers
Take Care of Your Prompt Bias! Investigating and Mitigating Prompt Bias in Factual Knowledge Extraction (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent research shows that pre-trained language models suffer from “prompt bias” in factual knowledge extraction. |
| Approach: | They propose a representation-based approach to mitigate prompt bias during inference time by querying the model and removing it from its internal representations to generate debiased representations. |
| Outcome: | The proposed approach corrects the overfitted performance caused by prompt bias and significantly improves prompt retrieval capability. |
Rethinking Prompt-based Debiasing in Large Language Model (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing prompt-based methods for debiasing are often superficial and lack a thorough understanding of complex bias concepts. |
| Approach: | They analyze a BBQ and stereoSet benchmarks to examine the assumption that large language models understand biases. |
| Outcome: | The proposed model misclassified 90% of unbiased content as biased despite high accuracy on BBQ dataset . the proposed model may have been flawed in previous attempts to debiase . |
Can Prompt Probe Pretrained Language Models? Understanding the Invisible Risks from a Causal View (2022.acl-long)
Copied to clipboard
| Challenge: | Recent studies have found prompt-based probing evaluations inaccurate, inconsistent and unreliable. |
| Approach: | They propose to conduct debiasing via causal intervention to uncover biases in probing evaluations . authors argue that prompt-based probing is inaccurate, inconsistent and unreliable . |
| Outcome: | This paper examines the effectiveness of prompt-based probing in pretrained language models . it highlights critical biases which could induce biased results and conclusions . authors suggest rethinking criteria for evaluating better pretrained models based on such evaluations . |
Social Bias Evaluation for Large Language Models Requires Prompt Variations (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies have tried to evaluate and mitigate social biases accurately using limited prompts. |
| Approach: | They investigate the sensitivity of Large Language Models when changing prompt variations . they found that LLM rankings fluctuate across prompts for both task performance and social bias . |
| Outcome: | The results show that LLM rankings fluctuate when changing prompt variations . |
Co2PT: Mitigating Bias in Pre-trained Language Models through Counterfactual Contrastive Prompt Tuning (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained language models can encode unfair social biases from large pre-training corpora and even amplify biase in downstream applications. |
| Approach: | They propose a *debias-while-prompt tuning* method for mitigating biases via counterfactual contrastive prompt tuning on downstream tasks. |
| Outcome: | The proposed method can mitigate biases on three extrinsic bias benchmarks and adapt to existing debiased language models. |
Challenging the Explanation Based on Preceding Tokens: Discovering Transferable Non-Literal Biasing (2026.acl-short)
Copied to clipboard
| Challenge: | et al. (2017) show that the generated preceding tokens may push the large language model towards the target answer. |
| Approach: | They find that generated preceding tokens may push large language models towards the target answer . they suggest that the LLM may intentionally use the semantically unrelated tokens to help generation of the target . |
| Outcome: | The generated preceding tokens may push the large language model towards the target answer . the biased connotations of the target response can also transfer to other prompts . |
Adaptive Prompting: Ad-hoc Prompt Composition for Social Bias Detection (2025.naacl-long)
Copied to clipboard
Maximilian Spliethöver, Tim Knebler, Fabian Fumagalli, Maximilian Muschalik, Barbara Hammer, Eyke Hüllermeier, Henning Wachsmuth
| Challenge: | Existing prompting techniques for large language models depend on several parameters, such as the task, language model, and context provided. |
| Approach: | They propose an adaptive prompting approach that predicts the optimal prompt composition ad-hoc for a given input. |
| Outcome: | The proposed approach ensures high detection performance and is best in several settings. |
Upstream Mitigation Is Not All You Need: Testing the Bias Transfer Hypothesis in Pre-Trained Language Models (2022.acl-long)
Copied to clipboard
| Challenge: | Large language models and other massively pre-trained "foundation" models can easily adapt to a wide variety of downstream tasks in a process called finetuning. |
| Approach: | They propose to use the bias transfer hypothesis to reduce social biases internalized by large language models during pre-training into harmful task-specific behavior after fine-tuning. |
| Outcome: | The bias transfer hypothesis is the theory that social biases internalized by large language models during pre-training transfer into harmful task-specific behavior after fine-tuning. |
When Do Pre-Training Biases Propagate to Downstream Tasks? A Case Study in Text Summarization (2023.eacl-main)
Copied to clipboard
Faisal Ladhak, Esin Durmus, Mirac Suzgun, Tianyi Zhang, Dan Jurafsky, Kathleen McKeown, Tatsunori Hashimoto
| Challenge: | Existing studies have shown that large language models contain linguistic and societal biases, but it is unclear how these biase amplify to downstream tasks. |
| Approach: | They investigate how name-nationality bias propagates from pre-training to downstream tasks . they show that these biases manifest themselves as hallucinations in summarization . |
| Outcome: | The proposed model can reduce the rate of hallucinations, but does not change the types of biases that do appear. |
On Measuring Social Biases in Prompt-Based Multi-Task Learning (2022.findings-naacl)
Copied to clipboard
| Challenge: | a large body of work within prompt engineering attempts to understand the effects of input forms and prompts in achieving superior performance. |
| Approach: | They propose a large-scale text-to-text language model trained using prompts . they consider two different forms of semantically equivalent inputs - question-answer format and premise-hypothesis format . |
| Outcome: | The proposed model can generalize into novel forms of language and handle novel tasks. |