“Thinking” Fair and Slow: On the Efficacy of Structured Prompts for Debiasing Language Models (2024.emnlp-main)
Copied to clipboard
Shaz Furniturewala, Surgan Jandial, Abhinav Java, Pragyan Banerjee, Simra Shahid, Sumit Bhatia, Kokil Jaidka
| Challenge: | Existing debiasing techniques are typically training-based or require access to the model’s internals and output distributions, so they are inaccessible to end-users looking to adapt LLM outputs for their particular needs. |
| Approach: | They propose a system-based iterative framework that uses System 2 thinking processes to induce logical, reflective, and critical text generation with single, multi-step, instruction, and role-based variants. |
| Outcome: | The proposed framework significantly improves over other frameworks demonstrating lower mean bias in the outputs with competitive performance on the downstream tasks. |
Similar Papers
Rethinking Prompt-based Debiasing in Large Language Model (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing prompt-based methods for debiasing are often superficial and lack a thorough understanding of complex bias concepts. |
| Approach: | They analyze a BBQ and stereoSet benchmarks to examine the assumption that large language models understand biases. |
| Outcome: | The proposed model misclassified 90% of unbiased content as biased despite high accuracy on BBQ dataset . the proposed model may have been flawed in previous attempts to debiase . |
FairSteer: Inference Time Debiasing for LLMs with Dynamic Activation Steering (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing prompt-based debiasing methods exhibit instability due to sensitivity to prompt changes . fine-tuning-based techniques incur substantial computational overhead and catastrophic forgetting . |
| Approach: | They propose a debiasing framework that encodes fairness-related features into separable directions in the hidden activation space. |
| Outcome: | The proposed framework performs inference-time debiasing without requiring retraining or prompt design . it detects bias signatures in activations and then computes debiased steering vectors . the proposed framework is available to download in the u.s. |
Cognitive Bias in Decision-Making with LLMs (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models inherit societal biases against protected groups and can be subject to functionally resembling cognitive bias. |
| Approach: | They propose a framework to uncover, evaluate, and mitigate cognitive bias in large language models by using a dataset containing 13,465 prompts to evaluate LLM decisions on different cognitive biases. |
| Outcome: | The proposed framework uncovers, evaluates, and mitigates cognitive bias in large language models, particularly in high-stakes decision-making tasks. |
Debiasing LLMs by Masking Unfairness-Driving Attention Heads (2026.findings-acl)
Copied to clipboard
Tingxu Han, Wei Song, Ziqi Ding, Ziming Li, Chunrong Fang, Yuekang Li, Dongfang Liu, Zhenyu Chen, Zhenting Wang
| Challenge: | Existing work probes when biased outputs appear, but gives little insight into the mechanisms that generate them, leaving existing mitigations largely fragile. |
| Approach: | They propose a lightweight debiasing framework that detects bias heads and selectively masks only those heads that activate under DA and CoT. |
| Outcome: | The proposed framework reduces unfairness by 391.9%- 534.5% in both one- and two-turn dialogues. |
DeCAP: Context-Adaptive Prompt Generation for Debiasing Zero-shot Question Answering in Large Language Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing zero-shot methods for Question Answering (QA) are efficient but fail to consider context and prevent bias propagation in the answers. |
| Approach: | They propose a method for debiasing Large Language Models using context-adaptive prompt generation that takes appropriate debiased actions based on the context and aNeutral Answer Guidance Generation to suppress the LLMs make objective judgments about the context. |
| Outcome: | The proposed method achieves state-of-the-art zero-shot debiased QA performance across eight LLMs. |
Do LLMs Overcome Shortcut Learning? An Evaluation of Shortcut Challenges in Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown remarkable capabilities in various tasks, but may rely on dataset biases as shortcuts for prediction. |
| Approach: | They propose to use a test suite to evaluate the impact of shortcuts on LLMs' performance. |
| Outcome: | The proposed test suite incorporates six shortcut types, five evaluation metrics, and four prompting strategies. |
More Samples or More Prompts? Exploring Effective Few-Shot In-Context Learning for LLMs with In-Context Sampling (2024.findings-naacl)
Copied to clipboard
Bingsheng Yao, Guiming Chen, Ruishi Zou, Yuxuan Lu, Jiachen Li, Shao Zhang, Yisi Sang, Sijia Liu, James Hendler, Dakuo Wang
| Challenge: | Existing studies on LLM prompting focus on selecting a better set of data samples inside one single prompt input, but why not design and leverage multiple ICL prompts together to further improve the LLM’s performance? |
| Approach: | They propose a low-resource LLM prompting technique to optimize the construction of multiple ICL prompt inputs to produce confident predictions. |
| Outcome: | The proposed technique can produce confident predictions by optimizing the construction of multiple ICL prompt inputs on four NLI datasets and one QA dataset. |
The Impossibility of Fair LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Existing frameworks for evaluating large language models do not extend to general-purpose AI contexts or are infeasible in practice. |
| Approach: | They analyze a variety of technical fairness frameworks to find inherent challenges . they find that each framework does not logically extend to the general-purpose AI context . |
| Outcome: | The proposed frameworks do not logically extend to the general-purpose AI context or are infeasible in practice due to large amounts of unstructured training data and potential combinations of human populations, use cases, and sensitive attributes. |
Co2PT: Mitigating Bias in Pre-trained Language Models through Counterfactual Contrastive Prompt Tuning (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained language models can encode unfair social biases from large pre-training corpora and even amplify biase in downstream applications. |
| Approach: | They propose a *debias-while-prompt tuning* method for mitigating biases via counterfactual contrastive prompt tuning on downstream tasks. |
| Outcome: | The proposed method can mitigate biases on three extrinsic bias benchmarks and adapt to existing debiased language models. |
Prompterator: Iterate Efficiently towards More Effective Prompts (2023.emnlp-demo)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) use a process known as prompting to solve arbitrary language tasks. prompting is a non-trivial task that requires experimentation in order to arrive at a prompt that solves a specific task. |
| Approach: | They propose a tool that helps users iterate over different potential prompts and choose the best performing one based on human feedback. |
| Outcome: | The proposed tool is open source and easily extensible. |