| Challenge: | a growing number of natural language processing models leave aside the language itself . a recent paradigm in the computational linguistics community is training models on specific perspectives of a segment of the population or an individual. |
| Approach: | They propose to use BERT-based classification models to detect stereotypes related to immigrants . they compare models with predictions from GPT-4 and annotated tweets from Spanish Twitter . |
| Outcome: | The proposed models are compared with predictions from the dataset of Spanish Twitter posts containing stereotypes . the models are confident in their predictions and more accurate for implicit stereotypes, the authors show . |
Similar Papers
Quantifying Stereotypes in Language (2024.eacl-long)
Copied to clipboard
| Challenge: | Existing studies define a sentence as stereotypical and anti-stereotypical, but they lack a fine-grained quantification of stereotypes. |
| Approach: | They quantify stereotypes in language by annotating a dataset to quantify stereotype of sentences. |
| Outcome: | The proposed models validate the findings of the current studies. |
Rethinking Research on Stereotypes: An Analysis through Social Psychological and Computational Perspectives (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing research on stereotypical biases ignores literature on them and results in resource wastage. |
| Approach: | They argue that stereotypes are social constructs shaping human perception and behavior that can produce harmful outcomes under specific conditions. |
| Outcome: | The proposed models can inherit and amplify stereotypes under certain conditions. |
Intersectional Stereotypes in Large Language Models: Dataset and Analysis (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies on intersectional stereotypes focus on broader, individual categories . current studies focus on single-group stereotypes, such as racial bias against African Americans . |
| Approach: | They propose to use a dataset of intersectional stereotypes curated with the ChatGPT model to analyze propagation in three contemporary LLMs. |
| Outcome: | The proposed dataset enables analysis of stereotype propagation in three contemporary LLMs. |
Analyzing Stereotypes in Generative Text Inference Tasks (2021.findings-acl)
Copied to clipboard
| Challenge: | Social psychology studies how social stereotypes are shared as part of cultural knowledge . |
| Approach: | They study how stereotypes manifest when potential targets are situated in neutral contexts . they collect human judgments on the presence of stereotypes in generated inferences based on annotator positionality . |
| Outcome: | The results show that the annotators' positions differ depending on the type of inferences they generate . |
Criteria for the Annotation of Implicit Stereotypes (2022.lrec-1)
Copied to clipboard
| Challenge: | social media has brought with it a massive channel for spreading and reinforcing stereotypes . most stereotypes are expressed implicitly and identifying them automatically remains a challenge . |
| Approach: | They propose criteria to facilitate the subjective task of identifying the presence of stereotypes . they propose a newsCom-Implicitness corpus of 1,911 sentences, of which 426 are explicit and implicit racial stereotypes. |
| Outcome: | The proposed criteria show that they obtain different inter-annotator agreement values . the criteria are applied to a corpus of 1,911 sentences, of which 426 are explicit and implicit racial stereotypes . |
Stepmothers are mean and academics are pretentious: What do pretrained language models learn about you? (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on "gender bias" and "racial bias" focus on stereotypical attributes of word representations . a new method to elicit stereotypical information is proposed to capture stereotypical traits in language models . |
| Approach: | They propose a method to elicit stereotypical information from pretrained language models . they use fine-tuning on news sources to study their emotional effects . |
| Outcome: | The proposed method can be used to analyze emotion and stereotype shifts due to linguistic experience using fine-tuning on news sources. |
SHADES: Towards a Multilingual Assessment of Stereotypes in Large Language Models (2025.naacl-long)
Copied to clipboard
Margaret Mitchell, Giuseppe Attanasio, Ioana Baldini, Miruna Clinciu, Jordan Clive, Pieter Delobelle, Manan Dey, Sil Hamilton, Timm Dill, Jad Doughman, Ritam Dutt, Avijit Ghosh, Jessica Zosa Forde, Carolin Holtermann, Lucie-Aimée Kaffee, Tanmay Laud, Anne Lauscher, Roberto L Lopez-Davila, Maraim Masoud, Nikita Nangia, Anaelia Ovalle, Giada Pistilli, Dragomir Radev, Beatrice Savoldi, Vipul Raheja, Jeremy Qin, Esther Ploeger, Arjun Subramonian, Kaustubh Dhole, Kaiser Sun, Amirbek Djanibekov, Jonibek Mansurov, Kayo Yin, Emilio Villa Cueva, Sagnik Mukherjee, Jerry Huang, Xudong Shen, Jay Gala, Hamdan Al-Ali, null Tair Djanibekov, Nurdaulet Mukhituly, Shangrui Nie, Shanya Sharma, Karolina Stanczak, Eliza Szczechla, Tiago Timponi Torrent, Deepak Tunuguntla, Marcelo Viridiano, Oskar Van Der Wal, Adina Yakefu, Aurélie Névéol, Mike Zhang, Sydney Zink, Zeerak Talat
| Challenge: | Large Language Models reproduce and exacerbate social biases present in training data, and resources to quantify this issue are limited. |
| Approach: | They propose a multilingual parallel dataset to examine culturally-specific stereotypes that may be learned by LLMs. |
| Outcome: | The proposed dataset includes stereotypes from 20 regions around the world and 16 languages, spanning multiple identity categories subject to discrimination worldwide. |
StereoDetect: Detecting Stereotypes and Anti-stereotypes the Correct Way Using Social Psychological Underpinnings (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Stereotypes are known to have harmful effects, making their detection critical . current research focuses on detecting and evaluating stereotypical biases . |
| Approach: | They propose a five-tuple definition and provide precise terminologies disentangling stereotypes, antistereotypes, stereotypical bias, and general bias. |
| Outcome: | The proposed framework disentangles stereotypes, antistereotypes, stereotypical bias, and general bias. |
Understanding and Countering Stereotypes: A Computational Approach to the Stereotype Content Model (2021.acl-long)
Copied to clipboard
| Challenge: | Stereotypical language expresses widely-held beliefs about different social categories. |
| Approach: | They propose a computational approach to interpreting stereotypes in text through the Stereotype Content Model (SCM), a comprehensive causal theory from social psychology. |
| Outcome: | The proposed model compares favourably with survey-based studies in the psychological literature on stereotypes and shows that it is realistic and effective. |
Blind Men and the Elephant: Diverse Perspectives on Gender Stereotypes in Benchmark Datasets (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing benchmarks for measuring gender stereotypical bias in language models are inconsistencies . lack of explicit standards in data gathering can have detrimental effects on results . |
| Approach: | They propose that currently available benchmarks capture only partial facets of gender stereotypes . they apply a framework from social psychology to balance data across components of gender stereotypes based on stereotypical benchmarks. |
| Outcome: | The proposed framework improves correlation between different benchmarks by using simple balancing techniques. |