| Challenge: | social media has brought with it a massive channel for spreading and reinforcing stereotypes . most stereotypes are expressed implicitly and identifying them automatically remains a challenge . |
| Approach: | They propose criteria to facilitate the subjective task of identifying the presence of stereotypes . they propose a newsCom-Implicitness corpus of 1,911 sentences, of which 426 are explicit and implicit racial stereotypes. |
| Outcome: | The proposed criteria show that they obtain different inter-annotator agreement values . the criteria are applied to a corpus of 1,911 sentences, of which 426 are explicit and implicit racial stereotypes . |
Similar Papers
Corpus Considerations for Annotator Modeling and Scaling (2024.naacl-long)
Copied to clipboard
| Challenge: | Recent trends in natural language processing and annotation tasks emphasize individual perspectives . annotator models that rely on a single ground truth may disregard valuable minority perspectives omissions . |
| Approach: | They propose a composite embedding approach to investigate annotator modeling techniques . they show that the commonly used user token model consistently outperforms more complex models . |
| Outcome: | The proposed model outperforms more complex models on a given dataset. |
Your Stereotypical Mileage May Vary: Practical Challenges of Evaluating Biases in Multiple Languages and Cultural Contexts (2024.lrec-main)
Copied to clipboard
Karen Fort, Laura Alonso Alemany, Luciana Benotti, Julien Bezançon, Claudia Borg, Marthese Borg, Yongjian Chen, Fanny Ducel, Yoann Dupont, Guido Ivetta, Zhijian Li, Margot Mieskes, Marco Naguib, Yuyan Qian, Matteo Radaelli, Wolfgang S. Schmeisser-Nieto, Emma Raimundo Schulz, Thiziri Saci, Sarah Saidi, Javier Torroba Marchante, Shilin Xie, Sergio E. Zanotto, Aurélie Névéol
| Challenge: | Recent studies have identified a gap in the availability of tools and resources to study bias in languages other than English and social contexts outside the north of America. |
| Approach: | They use stereotypes to build a corpus of sentence pairs that cover biases in seven cultural contexts. |
| Outcome: | The proposed resource covers a wide range of languages and cultural settings . it favors sentences that express stereotypes in most bias categories . |
Analyzing Stereotypes in Generative Text Inference Tasks (2021.findings-acl)
Copied to clipboard
| Challenge: | Social psychology studies how social stereotypes are shared as part of cultural knowledge . |
| Approach: | They study how stereotypes manifest when potential targets are situated in neutral contexts . they collect human judgments on the presence of stereotypes in generated inferences based on annotator positionality . |
| Outcome: | The results show that the annotators' positions differ depending on the type of inferences they generate . |
Quantifying Stereotypes in Language (2024.eacl-long)
Copied to clipboard
| Challenge: | Existing studies define a sentence as stereotypical and anti-stereotypical, but they lack a fine-grained quantification of stereotypes. |
| Approach: | They quantify stereotypes in language by annotating a dataset to quantify stereotype of sentences. |
| Outcome: | The proposed models validate the findings of the current studies. |
StereoDetect: Detecting Stereotypes and Anti-stereotypes the Correct Way Using Social Psychological Underpinnings (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Stereotypes are known to have harmful effects, making their detection critical . current research focuses on detecting and evaluating stereotypical biases . |
| Approach: | They propose a five-tuple definition and provide precise terminologies disentangling stereotypes, antistereotypes, stereotypical bias, and general bias. |
| Outcome: | The proposed framework disentangles stereotypes, antistereotypes, stereotypical bias, and general bias. |
Latent Hatred: A Benchmark for Understanding Implicit Hate Speech (2021.emnlp-main)
Copied to clipboard
Mai ElSherief, Caleb Ziems, David Muchlinski, Vaishnavi Anupindi, Jordyn Seybolt, Munmun De Choudhury, Diyi Yang
| Challenge: | Existing studies on explicit or overt hate speech have failed to address a more pervasive form based on coded or indirect language. |
| Approach: | They propose a theoretically-justified taxonomy of implicit hate speech and a benchmark corpus with fine-grained labels for each message and its implication. |
| Outcome: | The proposed dataset will serve as a useful benchmark for understanding this multifaceted issue. |
Extracting Age-Related Stereotypes from Social Media Texts (2022.lrec-1)
Copied to clipboard
| Challenge: | a method for extracting age-related stereotypes from Twitter data is under-studied in NLP . stereotyping on the basis of protected characteristics has been understudied . |
| Approach: | They propose a method for extracting age-related stereotypes from Twitter data . they generate a corpus of 300,000 over-generalizations about four contemporary generations . |
| Outcome: | The method uncovers common stereotypes as reported in media and psychological literature . it also finds that stereotypes for different generations vary across topics . |
Implicit Knowledge in Argumentative Texts: An Annotated Corpus (2020.lrec-1)
Copied to clipboard
| Challenge: | Especially in argumentative texts, people omit information that seems clear and evident . a computational system typically does not possess commonsense or domain-specific knowledge to reconstruct implied information. |
| Approach: | They build a corpus of human annotations of missing and implied information in argumentative texts. |
| Outcome: | The proposed dataset can help to assist automated argument analysis and guide the process of revealing implicit information in argumentative texts automatically. |
Stepmothers are mean and academics are pretentious: What do pretrained language models learn about you? (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on "gender bias" and "racial bias" focus on stereotypical attributes of word representations . a new method to elicit stereotypical information is proposed to capture stereotypical traits in language models . |
| Approach: | They propose a method to elicit stereotypical information from pretrained language models . they use fine-tuning on news sources to study their emotional effects . |
| Outcome: | The proposed method can be used to analyze emotion and stereotype shifts due to linguistic experience using fine-tuning on news sources. |
Human vs. Machine Perceptions on Immigration Stereotypes (2024.lrec-main)
Copied to clipboard
| Challenge: | a growing number of natural language processing models leave aside the language itself . a recent paradigm in the computational linguistics community is training models on specific perspectives of a segment of the population or an individual. |
| Approach: | They propose to use BERT-based classification models to detect stereotypes related to immigrants . they compare models with predictions from GPT-4 and annotated tweets from Spanish Twitter . |
| Outcome: | The proposed models are compared with predictions from the dataset of Spanish Twitter posts containing stereotypes . the models are confident in their predictions and more accurate for implicit stereotypes, the authors show . |