Adaptive Axes: A Pipeline for In-domain Social Stereotype Analysis (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to quantify social stereotypes have struggled to capture the variability in stereotypes across conceptual domains for the same social group. |
| Approach: | They propose to use text embedding models and adaptive semantic axes to recover stereotypes from contextual representations by using large language models. |
| Outcome: | The proposed pipeline surpasses token-based methods in capturing in-domain framing and tracks stereotypes along domain-specific semantic axes for in- domain texts. |
Similar Papers
Discovering Differences in the Representation of People using Contextualized Semantic Axes (2022.emnlp-main)
Copied to clipboard
| Challenge: | Past work has compared embeddings against “semantic axes” that represent two opposing concepts. |
| Approach: | They extend this paradigm to BERT embeddings and construct contextualized axes that mitigate pitfall where antonyms have neighboring representations. |
| Outcome: | The proposed axes can characterize differences among instances of the same word type on two people-centric datasets. |
A Comprehensive Framework to Operationalize Social Stereotypes for Responsible AI Evaluations (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent years have seen unprecedented gains in generative AI models' capabilities across modalitieslanguage, image, audio, and video domains across the globe. |
| Approach: | They propose a framework to operationalize stereotypes in generative AI evaluations using social psychological research and NLP data. |
| Outcome: | The proposed framework identifies key components of stereotypes that are crucial in AI evaluation, including the target group, associated attribute, relationship characteristics, perceiving group, and context. |
Rethinking Research on Stereotypes: An Analysis through Social Psychological and Computational Perspectives (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing research on stereotypical biases ignores literature on them and results in resource wastage. |
| Approach: | They argue that stereotypes are social constructs shaping human perception and behavior that can produce harmful outcomes under specific conditions. |
| Outcome: | The proposed models can inherit and amplify stereotypes under certain conditions. |
Understanding and Countering Stereotypes: A Computational Approach to the Stereotype Content Model (2021.acl-long)
Copied to clipboard
| Challenge: | Stereotypical language expresses widely-held beliefs about different social categories. |
| Approach: | They propose a computational approach to interpreting stereotypes in text through the Stereotype Content Model (SCM), a comprehensive causal theory from social psychology. |
| Outcome: | The proposed model compares favourably with survey-based studies in the psychological literature on stereotypes and shows that it is realistic and effective. |
Extracting Age-Related Stereotypes from Social Media Texts (2022.lrec-1)
Copied to clipboard
| Challenge: | a method for extracting age-related stereotypes from Twitter data is under-studied in NLP . stereotyping on the basis of protected characteristics has been understudied . |
| Approach: | They propose a method for extracting age-related stereotypes from Twitter data . they generate a corpus of 300,000 over-generalizations about four contemporary generations . |
| Outcome: | The method uncovers common stereotypes as reported in media and psychological literature . it also finds that stereotypes for different generations vary across topics . |
Intersectional Stereotypes in Large Language Models: Dataset and Analysis (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies on intersectional stereotypes focus on broader, individual categories . current studies focus on single-group stereotypes, such as racial bias against African Americans . |
| Approach: | They propose to use a dataset of intersectional stereotypes curated with the ChatGPT model to analyze propagation in three contemporary LLMs. |
| Outcome: | The proposed dataset enables analysis of stereotype propagation in three contemporary LLMs. |
StereoMap: Quantifying the Awareness of Human-like Stereotypes in Large Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have been observed to encode harmful associations present in the training data. |
| Approach: | They propose a framework to map LLMs' perceptions of how demographic groups have been viewed by society using the dimensions of Warmth and Competence. |
| Outcome: | The proposed framework maps LLMs’ perceptions of social groups using the dimensions of Warmth and Competence. |
Theory-Grounded Measurement of U.S. Social Stereotypes in English Language Models (2022.naacl-main)
Copied to clipboard
| Challenge: | Pre-trained language models encode correlations between social groups and traits, like associating the group with the group. |
| Approach: | They adapt the Agency-Belief-Communion (ABC) stereotype model to a language model and introduce the sensitivity test (SeT) to measure stereotypical associations. |
| Outcome: | The proposed framework is used to measure stereotyping of intersectional identities in language models. |
Quantifying Stereotypes in Language (2024.eacl-long)
Copied to clipboard
| Challenge: | Existing studies define a sentence as stereotypical and anti-stereotypical, but they lack a fine-grained quantification of stereotypes. |
| Approach: | They quantify stereotypes in language by annotating a dataset to quantify stereotype of sentences. |
| Outcome: | The proposed models validate the findings of the current studies. |
Toward Inclusive Language Models: Sparsity-Driven Calibration for Systematic and Interpretable Mitigation of Social Biases in LLMs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a new method to mitigate stereotypical bias in large language models is needed . inherent biases from training on vast Internet datasets can amplify harmful stereotypes . |
| Approach: | They propose a method to identify stereotypical bias in decoder-only transformer models . they apply a localization mechanism that correlates internal activations with a new Context Influence score . |
| Outcome: | The proposed method reduces stereotypical biases on BBQ, StereoSet, and CrowS-Pairs while improving reasoning performance on MMLU by 10%. |