Papers by Mahdi Zakizadeh
Blind Men and the Elephant: Diverse Perspectives on Gender Stereotypes in Benchmark Datasets (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing benchmarks for measuring gender stereotypical bias in language models are inconsistencies . lack of explicit standards in data gathering can have detrimental effects on results . |
| Approach: | They propose that currently available benchmarks capture only partial facets of gender stereotypes . they apply a framework from social psychology to balance data across components of gender stereotypes based on stereotypical benchmarks. |
| Outcome: | The proposed framework improves correlation between different benchmarks by using simple balancing techniques. |
DiFair: A Benchmark for Disentangled Assessment of Gender Knowledge and Bias (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to mitigate gender bias in pre-trained language models are often evaluated on datasets that check the extent to which the model is gender-neutral in its predictions. |
| Approach: | They propose to use a manually curated dataset to measure gender bias and to measure useful gender knowledge. |
| Outcome: | The proposed dataset aims to quantify gender biases and to assess their impact on useful gender knowledge. |