Real Men are Tough: Evaluating Gender Bias and Sensitivity to Masculinity Norms in LLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models exhibit gender bias, but most evaluations focus on downstream stereotypes . a recent study found that explicit endorsement of masculinity norms is low across models . |
| Approach: | They investigate whether large language models rely on traditional masculinity norms as latent priors in gender-biased inference. |
| Outcome: | The findings show that large language models rely on stereotypes as latent priors . the authors used the Male Role Norms Inventory (MRNI) to investigate gender bias . |
Similar Papers
Gender Bias in Decision-Making with Large Language Models: A Study of Relationship Conflicts (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models acquire beliefs about gender from training data and can therefore generate text with stereotypical gender attitudes. |
| Approach: | They use a decision-making lens to examine gender equity within large language models . they explore relationships through typical and gender-neutral names . |
| Outcome: | The proposed model generation and classification models exhibit stereotypical gender biases . the proposed model generates gender-neutral names, with and without safety enhancements, and egalitarian versus traditional scenarios across topics. |
A Comparative Study of Explicit and Implicit Gender Biases in Large Language Models via Self-evaluation (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing studies on the explicit and implicit biases in large language models (LLMs) focus on either explicit or implicit bias. |
| Approach: | They propose a self-evaluation-based two-stage measurement of explicit and implicit biases within large language models grounded in social psychology. |
| Outcome: | The proposed model is based on two stages of self-evaluation on state-of-the-art LLMs to measure explicit bias toward social targets, where bias is less likely to be self-recognized by the LLM. |
Evaluating Gender Bias of LLMs in Making Morality Judgements (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown remarkable capabilities in a multitude of NLP tasks, but are still not immune to limitations such as gender bias. |
| Approach: | They propose to use a dataset to examine whether LLMs possess gender bias when asked to give moral opinions. |
| Outcome: | The proposed models show that they are biased when asked to give moral opinions. |
Measuring Bias or Measuring the Task: Understanding the Brittle Nature of LLM Gender Biases (2025.emnlp-main)
Copied to clipboard
| Challenge: | a growing number of efforts to measure and mitigate gender bias have focused on task prompts that overtly or covertly signal the presence of gender bias-related content. |
| Approach: | They examine how signaling the evaluative purpose of a task impacts measured gender bias in LLMs. |
| Outcome: | The proposed models show that prompts that align with (gender bias) evaluation framing elicit distinct gender output distributions compared to less evaluation-framed prompts. |
Beyond Names: How Grammatical Gender Markers Bias LLM-based Educational Recommendations (2026.eacl-long)
Copied to clipboard
| Challenge: | grammatical gender cues alone trigger substantial distributional shifts in educational recommendations . authors show that up to 76% of the bias exhibited when using prompts with proper names is already present with grammatical gender markers alone. |
| Approach: | They investigate gender biases exhibited by LLM-based virtual assistants in Italian . they show that simply changing noun and adjective endings significantly shifts recommendations . |
| Outcome: | The findings highlight the need for robust bias evaluation and mitigation strategies before deploying LLM-based virtual assistants in student-facing contexts. |
Do LLMs Align Human Values Regarding Social Biases? Judging and Explaining Social Biases with LLMs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models can lead to undesired consequences when misaligned with human values . previous studies have shown misalignment of LLMs with human value using expert-designed or agent-based emulated bias scenarios . |
| Approach: | They investigate whether large language models (LLMs) are misaligned with human values . they find no significant differences in understanding of HVSB between LLMs . |
| Outcome: | The results show that large language models do not have lower misalignment rates and attack success rates . the study also shows that smaller language models have the ability to explain HVSB . |
“Kelly is a Warm Person, Joseph is a Role Model”: Gender Biases in LLM-Generated Reference Letters (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are an effective tool to assist individuals in writing documents. |
| Approach: | They examine gender biases in large language models (LLMs)-generated reference letters . they find that models are biased because they are hallucinated . |
| Outcome: | The proposed model-generated reference letters are evaluated on 2 popular LLMs- ChatGPT and Alpaca. |
QueerGen: How LLMs Reflect Societal Norms on Gender and Sexuality in Sentence Completion Task (2026.findings-eacl)
Copied to clipboard
| Challenge: | Autoregressive Language Models (ARLMs) partially mitigate these patterns, while closed-access ARLMs tend to produce more harmful outputs for unmarked subjects. |
| Approach: | They examine whether explicit information about a subject’s gender or sexuality influences LLM responses across three subject categories: queer-marked, non-queer-mark, and the normalized "unmarked" category. |
| Outcome: | The proposed models reproduce normative social assumptions, but the form and degree of bias depend on model characteristics, which may redistribute—but not eliminate—representational harms. |
Bias in Language Models: Beyond Trick Tests and Towards RUTEd Evaluation (2025.acl-long)
Copied to clipboard
| Challenge: | Standard bias benchmarks are used for large language models to measure the association between social attributes and single-word outputs. |
| Approach: | They adapt three standard bias metrics of next-word prediction to measure gender-occupation bias and develop an analogous RUTEd evaluation in three contexts of real-world LLM use. |
| Outcome: | The proposed benchmarks are robust to lengthening model outputs via a more realistic user prompt in the domain of gender-occupation bias. |
Hire Me or Not? Examining Language Model’s Behavior with Occupation Attributes (2025.coling-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have been widely integrated into production pipelines due to their impressive performance across multiple tasks. |
| Approach: | They construct a dataset using a standard occupation classification knowledge base and tested it on three families of LLMs. |
| Outcome: | The proposed framework analyzes LLMs’ behavior with respect to gender stereotypes in the context of occupation decision making. |