WIKIBIAS: Detecting Multi-Span Subjective Biases in Language (2021.findings-emnlp)
Copied to clipboard
| Challenge: | a particular type of bias is subjective bias, which introduces improper attitudes or presents a statement with the presupposition of truth. |
| Approach: | They propose to annotate a Wikipedia edits corpus with 4,000 sentence pairs to detect subjective bias. |
| Outcome: | The proposed dataset can be used as a research benchmark and generalize to multiple domains. |
Similar Papers
Mind Your Bias: A Critical Review of Bias Detection Methods for Contextual Language Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for detection of biases in contextual language models are inconsistent and inconclusive. |
| Approach: | They propose to use word embedding association test to detect biases in contextual language models to compare them with other methods. |
| Outcome: | The proposed methods are inconsistent and inconclusive for language models with word embeddings. |
Disentangling Structure and Style: Political Bias Detection in News by Inducing Document Hierarchy (2023.findings-emnlp)
Copied to clipboard
| Challenge: | a new method to detect political bias in news articles overcomes this domain dependency . partisan bias exists in various social issues, including the 2016 presidential election . |
| Approach: | They propose a multi-head hierarchical attention model that encodes the structure of long documents through a diverse ensemble of attention heads. |
| Outcome: | The proposed model outperforms existing methods for detecting political bias in news articles. |
Annotating and Analyzing Biased Sentences in News Articles using Crowdsourcing (2020.lrec-1)
Copied to clipboard
| Challenge: | a lack of publicly available news bias datasets has hindered efforts to detect subtle biases in news articles. |
| Approach: | They propose a news bias dataset which contains sentences with bias labels . they propose to use the dataset to develop and evaluate methods for detecting news bias . |
| Outcome: | The proposed dataset can be used for analyzing news bias and for developing and evaluating methods for news bias detection. |
Multi-Modal Bias: Introducing a Framework for Stereotypical Bias Assessment beyond Gender and Race in Vision–Language Models (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent advances in self-supervised training have led to a new class of pretrained vision–language models. |
| Approach: | They propose a visual and textual bias benchmark to assess bias in self-supervised multimodal models using 3,800 images and phrases from 14 population subgroups. |
| Outcome: | The proposed model shows that it favors certain groups while maintaining the accuracy of the model. |
Discovering Biased News Articles Leveraging Multiple Human Annotations (2020.lrec-1)
Copied to clipboard
| Challenge: | Political propaganda and one-sided views can be found in the news and can cause distrust in media. |
| Approach: | They propose to annotate politically biased news articles by an algorithm annotated by domain experts and crowd workers and to compare them to crowd workers. |
| Outcome: | The proposed method compares domain experts to crowd workers and shows that bias can be detected automatically. |
The Tail Wagging the Dog: Dataset Construction Biases of Social Bias Benchmarks (2023.acl-short)
Copied to clipboard
| Challenge: | omnipresence of large pre-trained language models has fueled concerns regarding systematic biases carried over from underlying data into the applications they are used in. |
| Approach: | They propose to compare social biases with non-social biase masked by alternate constructions that maintain the essence of their social bias. |
| Outcome: | The proposed benchmarks underestimate or overestimate the social bias in a given model. |
RedditBias: A Real-World Resource for Bias Evaluation and Debiasing of Conversational Language Models (2021.acl-long)
Copied to clipboard
| Challenge: | Recent work has focused on measuring and mitigating bias in pretrained language models. |
| Approach: | They propose a dataset that measures and mitigates bias across gender,race, religion, and queerness . they compare REDDITBIAS to a widely used conversational DialoGPT model . |
| Outcome: | The proposed framework measures and mitigates bias across gender,race, religion, and queerness dimensions. |
Sentence-level Media Bias Analysis Informed by Discourse Structures (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent work on detecting media bias at the level of individual articles is limited to single sentences. |
| Approach: | They propose to use a news discourse structure and PDTB discourse relations to identify bias sentences within an article that can illuminate and explain the overall bias of the entire article. |
| Outcome: | The proposed model can detect bias at the level of individual articles and a single sentence can explain it. |
Investigating Bias in LLM-Based Bias Detection: Disparities between LLMs and Human Perception (2025.coling-main)
Copied to clipboard
| Challenge: | Detecting media bias is critical due to the spread of misinformation and disinformation on social media platforms. |
| Approach: | They investigate the presence and nature of bias within large language models and its consequential impact on media bias detection. |
| Outcome: | The proposed debiasing strategies include prompt engineering and model fine-tuning. |
Multilingual Twitter Corpus and Baselines for Evaluating Demographic Bias in Hate Speech Recognition (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing work on document classification models mainly uses synthetic monolingual data without ground truth for author demographic attributes. |
| Approach: | They assemble and publish a multilingual Twitter corpus for the task of hate speech detection using inferred author demographic factors. |
| Outcome: | The results show that the classifiers learn human biases and can be discriminatory towards certain demographic groups. |