| Challenge: | Existing word embeddings have been used to study gender stereotypes in texts . however, evaluating their validities is still an open problem . et al.: this study investigates gender bias using the lens of language, especially, the words . |
| Approach: | They use word association test to derive bias scores for large amount of words . they find that these bias scores correlate well with bias in the real world . |
| Outcome: | The proposed method correlates well with bias in the real world, and with census data, it provides a different perspective on gender stereotypes in words. |
Similar Papers
Robustness and Reliability of Gender Bias Assessment in Word Embeddings: The Role of Base Pairs (2020.aacl-main)
Copied to clipboard
| Challenge: | Existing methods to quantify gender bias in word embeddings are not robust and cannot identify common types of bias. |
| Approach: | They propose to quantify gender bias by using cosine similarity to a pair of gender words and using analogies. |
| Outcome: | The proposed methods are not robust and cannot identify common types of bias, while analogies are unsuitable indicators. |
Assessing the Reliability of Word Embedding Gender Bias Measures (2021.emnlp-main)
Copied to clipboard
| Challenge: | Various measures have been proposed to quantify human-like social biases in word embeddings, but they can suffer from measurement error. |
| Approach: | They propose to assess the reliability of word embedding gender bias measures by examining their reliability across different choices of random seeds, scoring rules and words. |
| Outcome: | The proposed measures can suffer from measurement error, and the results inform better design of word embedding gender bias measures. |
When do Word Embeddings Accurately Reflect Surveys on our Beliefs About People? (2020.acl-main)
Copied to clipboard
| Challenge: | a study of word embeddings shows that social biases are more accurate than survey data for some dimensions of meaning. |
| Approach: | a new study investigates the extent to which word embeddings accurately reflect biases . they find that biased word embeds mirror survey data across 17 dimensions of social meaning . |
| Outcome: | a new study shows that word embeddings accurately reflect biases on average across dimensions of social meaning . biased embedders are more reflective of survey data for some dimensions of meaning than others, the study finds . |
Examining Gender Bias in Languages with Grammatical Gender (D19-1)
Copied to clipboard
| Challenge: | Existing studies on gender bias in word embeddings focus on English . however, these studies cannot be extended to languages with morphological agreement on gender . |
| Approach: | They propose new metrics to evaluate gender bias in word embeddings of English and Spanish . they extend existing approaches to mitigate gender bias while preserving original embeddables . |
| Outcome: | The proposed methods reduce gender bias while preserving the original embeddings. |
The Undesirable Dependence on Frequency of Gender Bias Metrics Based on Word Embeddings (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies have found word embeddings can capture semantic similarity but may be affected by word frequency. |
| Approach: | They find that word embeddings can capture semantic similarity but may be affected by word frequency . they compare this effect with an alternative metric based on Pointwise Mutual Information . |
| Outcome: | The proposed method does not depend on word frequency, but it does return female bias in low frequency words. |
Understanding Undesirable Word Embedding Associations (P19-1)
Copied to clipboard
| Challenge: | Word embeddings are often criticized for capturing undesirable word associations such as gender stereotypes. |
| Approach: | They propose to use subspace projection to debias vectors post hoc using a model that implicitly does matrix factorization to debunk gender bias. |
| Outcome: | The proposed test overestimates gender bias in word embeddings by using subspace projection, a method that is widely used in training. |
Automatically Inferring Gender Associations from Language (D19-1)
Copied to clipboard
| Challenge: | In this paper, we demonstrate that there are large-scale differences in the ways that people talk about women and men and that these differences vary across domains. |
| Approach: | They propose to integrate two datasets and a novel approach to automatically infer gender associations from language and find coherent word clusters and label clusters for the semantic concepts they represent. |
| Outcome: | The proposed methods outperform strong baselines in large-scale studies of how people talk about women and men in two different settings. |
Unequal Representations: Analyzing Intersectional Biases in Word Embeddings Using Representational Similarity Analysis (2020.coling-main)
Copied to clipboard
| Challenge: | Specifically, we probe contextualized and non-contextualized word embeddings for evidence of intersectional biases against Black women. |
| Approach: | They propose a representational similarity analysis approach to detect human-like biases in word embeddings using representational similarities analysis. |
| Outcome: | The proposed approach aligns with intersectionality theory, which states that multiple identity categories layer on top of each other to create unique modes of discrimination that are not shared by any individual category. |
Gender Stereotypes Differ between Male and Female Writings (P19-2)
Copied to clipboard
| Challenge: | a new study quantitatively evaluates gender stereotypes in written language . female writings contain fewer gender stereotype scores than male writings . |
| Approach: | They quantitatively evaluate and analyze gender stereotypes in written language . they compare writings by female authors with writings from male authors . |
| Outcome: | The results show that writings by female authors have lower gender stereotype scores . the authors plan on using more datasets over the past century to study gender stereotypes . |
Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Them (N19-1)
Copied to clipboard
| Challenge: | Existing methods to remove gender bias from word embeddings are insufficient, we argue . existing methods for gender-neutral modeling are ineffective, we conclude . |
| Approach: | They propose methods to reduce gender bias in word embeddings by debiasing them using text corpora. |
| Outcome: | The proposed methods show that they can reduce gender bias in word embeddings . the proposed methods are insufficient and should not be trusted, the authors argue . |