Mind the Biases: Quantifying Cognitive Biases in Language Model Prompting (2023.findings-acl)
Copied to clipboard
| Challenge: | Cognitive biases in the human decision making process can lead to flawed responses when we are under uncertainty. |
| Approach: | They propose to expose cognitive biases on results of language model prompting which display bias modes resembling cognitive bias. |
| Outcome: | The proposed methods show that a toning-down transformation of the drug-drug description in a prompt can elicit a bias similar to the framing effect, warning users to distrust when prompting language models for answers. |
Similar Papers
Beyond Performance: Quantifying and Mitigating Label Bias in LLMs (2024.naacl-long)
Copied to clipboard
| Challenge: | Large language models exhibit undesirable preference toward predicting certain answers over others, despite their adaptability to diverse tasks. |
| Approach: | They propose a label bias calibration method that outperforms recent calibration approaches for improving performance and mitigating label bias. |
| Outcome: | The proposed method outperforms calibration approaches for improving performance and mitigating label bias. |
Take Care of Your Prompt Bias! Investigating and Mitigating Prompt Bias in Factual Knowledge Extraction (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent research shows that pre-trained language models suffer from “prompt bias” in factual knowledge extraction. |
| Approach: | They propose a representation-based approach to mitigate prompt bias during inference time by querying the model and removing it from its internal representations to generate debiased representations. |
| Outcome: | The proposed approach corrects the overfitted performance caused by prompt bias and significantly improves prompt retrieval capability. |
Cognitive Bias in Decision-Making with LLMs (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models inherit societal biases against protected groups and can be subject to functionally resembling cognitive bias. |
| Approach: | They propose a framework to uncover, evaluate, and mitigate cognitive bias in large language models by using a dataset containing 13,465 prompts to evaluate LLM decisions on different cognitive biases. |
| Outcome: | The proposed framework uncovers, evaluates, and mitigates cognitive bias in large language models, particularly in high-stakes decision-making tasks. |
Analyzing the Limits of Self-Supervision in Handling Bias in Language (2022.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study shows that natural language models can perform tasks with little to no in-context supervision . a number of tasks are performed using self-supervised pre-training . |
| Approach: | They define and comprehensively evaluate how well natural language taskprompting captures the semantics of four tasks for bias: diagnosis, identification, extraction and rephrasing. |
| Outcome: | The proposed model performs to wide varying degrees across bias dimensions . the model is largely challenged when prompted to perform these tasks . |
Social Bias Evaluation for Large Language Models Requires Prompt Variations (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies have tried to evaluate and mitigate social biases accurately using limited prompts. |
| Approach: | They investigate the sensitivity of Large Language Models when changing prompt variations . they found that LLM rankings fluctuate across prompts for both task performance and social bias . |
| Outcome: | The results show that LLM rankings fluctuate when changing prompt variations . |
Rethinking Prompt-based Debiasing in Large Language Model (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing prompt-based methods for debiasing are often superficial and lack a thorough understanding of complex bias concepts. |
| Approach: | They analyze a BBQ and stereoSet benchmarks to examine the assumption that large language models understand biases. |
| Outcome: | The proposed model misclassified 90% of unbiased content as biased despite high accuracy on BBQ dataset . the proposed model may have been flawed in previous attempts to debiase . |
This prompt is measuring <mask>: evaluating bias evaluation in language models (2023.findings-acl)
Copied to clipboard
| Challenge: | a growing body of work uses prompts and templates to assess bias in language models . authors examine the scope of possible bias types and identify those under-researched . |
| Approach: | They draw on a measurement modelling framework to create a bias taxonomy . they show that bias tests are often unstated or ambiguous, carry implicit assumptions . |
| Outcome: | The proposed taxonomy shows that bias tests are often unstated or ambiguous . the analysis illuminates the scope of possible bias types the field can measure . |
Bias Beware: The Impact of Cognitive Biases on LLM-Driven Product Recommendations (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have revolutionized product recommenders, but their susceptibility to adversarial manipulations is difficult to detect. |
| Approach: | They propose to use large language models to investigate cognitive biases as adversarial strategies in product research using LLMs. |
| Outcome: | The proposed approach is the first to tap into human psychological principles, making such manipulations hard to detect. |
Measuring Bias or Measuring the Task: Understanding the Brittle Nature of LLM Gender Biases (2025.emnlp-main)
Copied to clipboard
| Challenge: | a growing number of efforts to measure and mitigate gender bias have focused on task prompts that overtly or covertly signal the presence of gender bias-related content. |
| Approach: | They examine how signaling the evaluative purpose of a task impacts measured gender bias in LLMs. |
| Outcome: | The proposed models show that prompts that align with (gender bias) evaluation framing elicit distinct gender output distributions compared to less evaluation-framed prompts. |
What Did I Do Wrong? Quantifying LLMs’ Sensitivity and Consistency to Prompt Engineering (2025.naacl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have significantly improved productivity in a number of routine tasks. |
| Approach: | They propose two metrics for classification tasks, namely *sensitivity* and *consistency*, which are complementary to task performance. |
| Outcome: | The proposed metrics are complementary to task performance and can be used to guide prompt engineering and obtain LLMs that balance robustness and performance. |