Investigating Sports Commentator Bias within a Large Corpus of American Football Broadcasts (D19-1)
Copied to clipboard
| Challenge: | a recent study shows that sports broadcasters build drama into play-by-play commentary by building team and player narratives through subjective analyses and anecdotes. |
| Approach: | They use FOOTBALL to examine racial bias in sports commentary . they identify major confounding factors for researchers examining rraecial bias . |
| Outcome: | The proposed dataset supports previous social science studies on commentator bias . it contains 1,455 broadcast football transcripts annotated with 250K player mentions and racial metadata . |
Similar Papers
The Risk of Racial Bias in Hate Speech Detection (P19-1)
Copied to clipboard
| Challenge: | Annotators’ insensitivity to differences in dialect can lead to racial bias in automatic hate speech detection models, potentially amplifying harm against minority populations. |
| Approach: | They propose *dialect* and *race priming* as ways to reduce the racial bias in hate speech detection models by detecting differences in dialects in annotated tweets. |
| Outcome: | The proposed models acquire and propagate these biases, such that AAE tweets and tweets by self-identified African Americans are up to two times more likely to be labelled as offensive compared to others. |
The Tail Wagging the Dog: Dataset Construction Biases of Social Bias Benchmarks (2023.acl-short)
Copied to clipboard
| Challenge: | omnipresence of large pre-trained language models has fueled concerns regarding systematic biases carried over from underlying data into the applications they are used in. |
| Approach: | They propose to compare social biases with non-social biase masked by alternate constructions that maintain the essence of their social bias. |
| Outcome: | The proposed benchmarks underestimate or overestimate the social bias in a given model. |
WIKIBIAS: Detecting Multi-Span Subjective Biases in Language (2021.findings-emnlp)
Copied to clipboard
| Challenge: | a particular type of bias is subjective bias, which introduces improper attitudes or presents a statement with the presupposition of truth. |
| Approach: | They propose to annotate a Wikipedia edits corpus with 4,000 sentence pairs to detect subjective bias. |
| Outcome: | The proposed dataset can be used as a research benchmark and generalize to multiple domains. |
Hollywood Identity Bias Dataset: A Context Oriented Bias Analysis of Movie Dialogues (2022.lrec-1)
Copied to clipboard
Sandhya Singh, Prapti Roy, Nihar Sahoo, Niteesh Mallela, Himanshu Gupta, Pushpak Bhattacharyya, Milind Savagaonkar, Nidhi Sultan, Roshni Ramnani, Anutosh Maitra, Shubhashis Sengupta
| Challenge: | Movies reflect society and also hold power to transform opinions. |
| Approach: | They propose to annotate movie scripts for identity bias using a dataset that is annotated for gender, race/ethnicity, religion, age, occupation, LGBTQ, and other . |
| Outcome: | The proposed dataset contains dialogue turns annotated for gender, race/ethnicity, religion, age, occupation, LGBTQ, and other, which contains biases like body shaming, personality bias, etc. |
Bias and Fairness in Natural Language Processing (D19-2)
Copied to clipboard
| Challenge: | a tutorial will review the history of bias and fairness studies in machine learning and language processing . |
| Approach: | This tutorial reviews the history of bias and fairness studies in machine learning and language processing . it presents recent community effort to quantify and mitigat bias in natural language processing models . |
| Outcome: | This tutorial reviews the history of bias and fairness studies in machine learning and language processing . it aims to quantify and mitigate bias in natural language processing models for a wide spectrum of tasks . |
Mind Your Bias: A Critical Review of Bias Detection Methods for Contextual Language Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for detection of biases in contextual language models are inconsistent and inconclusive. |
| Approach: | They propose to use word embedding association test to detect biases in contextual language models to compare them with other methods. |
| Outcome: | The proposed methods are inconsistent and inconclusive for language models with word embeddings. |
Sports and Women’s Sports: Gender Bias in Text Generation with Olympic Data (2025.naacl-short)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) generate text that is stereotypical or not representative of the viewpoints and values of historically marginalized demographic groups. |
| Approach: | They propose to use data from the Olympic Games to investigate gender bias in large language models. |
| Outcome: | The proposed model consistently biased against women when the gender is ambiguous in the prompt, revealing pervasive gender bias in LLMs in the context of athletics. |
A Survey of Race, Racism, and Anti-Racism in NLP (2021.acl-long)
Copied to clipboard
| Challenge: | despite inextricable ties between race and language, little work has considered race in NLP research and development. |
| Approach: | They survey 79 papers from the ACL anthology that mention race . they find race has been siloed as a niche topic and ignored in many NLP tasks . authors call for inclusion and racial justice in NLP research practices . |
| Outcome: | The findings highlight the need for inclusion and racial justice in NLP research practices. |
RedditBias: A Real-World Resource for Bias Evaluation and Debiasing of Conversational Language Models (2021.acl-long)
Copied to clipboard
| Challenge: | Recent work has focused on measuring and mitigating bias in pretrained language models. |
| Approach: | They propose a dataset that measures and mitigates bias across gender,race, religion, and queerness . they compare REDDITBIAS to a widely used conversational DialoGPT model . |
| Outcome: | The proposed framework measures and mitigates bias across gender,race, religion, and queerness dimensions. |
From Prejudice to Parity: A New Approach to Debiasing Large Language Model Word Embeddings (2025.coling-main)
Copied to clipboard
| Challenge: | Existing work in this field has looked most commonly into gender bias, racial bias, and religious bias. |
| Approach: | They propose an algorithm that uses a neural network to perform ‘soft debiasing’ and build on the seminal work of (CITATION) and (CitATION). |
| Outcome: | The proposed algorithm outperforms current methods on gender, race, and religion metrics on a wide range of metrics. |