MAGPIE: Multi-Task Analysis of Media-Bias Generalization with Pre-Trained Identification of Expressions (2024.lrec-main)
Copied to clipboard
Tomáš Horych, Martin Paul Wessel, Jan Philip Wahle, Terry Ruas, Jerome Waßmuth, André Greiner-Petter, Akiko Aizawa, Bela Gipp, Timo Spinde
| Challenge: | Existing approaches to media bias detection lack generalizability, resulting in limited generalizarability. |
| Approach: | They propose a large-scale multi-task pre-training approach specifically tailored for media bias detection that can be used to train 59 bias-related tasks. |
| Outcome: | The proposed approach outperforms existing methods on the BABE dataset with a relative improvement of 3.3% F1-score. |
Similar Papers
Neural Media Bias Detection Using Distant Supervision With BABE - Bias Annotations By Experts (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies on the detection and aggregation of media bias lack a gold standard data set and high context dependencies. |
| Approach: | They propose to use a data set to identify media bias by word and sentence level . they propose to train a model to detect bias-inducing sentences in news articles automatically . |
| Outcome: | The proposed model outperforms existing methods on a large corpus of labels on the word and sentence level. |
Media Bias Detection Across Families of Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Traditional NLP models have shown good performance in classifying media bias, but require careful model design and extensive tuning. |
| Approach: | They ask how well prompting of large language models can recognize media bias. |
| Outcome: | The prompt-based models deliver comparable performance to traditional models with greatly reduced effort and the availability of context substantially improves results. |
The Promises and Pitfalls of LLM Annotations in Dataset Labeling: a Case Study on Media Bias Detection (2025.findings-naacl)
Copied to clipboard
Tomáš Horych, Christoph Mandl, Terry Ruas, Andre Greiner-Petter, Bela Gipp, Akiko Aizawa, Timo Spinde
| Challenge: | Recent research suggests using Large Language Models (LLMs) to automate the annotation process, reducing these costs while maintaining data quality. |
| Approach: | They propose to use Large Language Models to automate annotation process and train classifiers on large datasets. |
| Outcome: | The proposed model outperforms all of the annotator LLMs on two media bias benchmark datasets (BABE and BASIL) while maintaining data quality. |
IndiVec: An Exploration of Leveraging Large Language Models for Media Bias Detection with Fine-Grained Bias Indicators (2024.findings-eacl)
Copied to clipboard
| Challenge: | Existing studies on social media bias detection focus on fine-tuning models specific to particular datasets and testing them on corresponding test sets. |
| Approach: | They propose a general bias detection framework, IndiVec, built upon large language models and vector databases. |
| Outcome: | The proposed framework outperforms baseline methods on four political bias datasets and provides explicit top-k indicators to interpret bias predictions. |
Investigating Bias in LLM-Based Bias Detection: Disparities between LLMs and Human Perception (2025.coling-main)
Copied to clipboard
| Challenge: | Detecting media bias is critical due to the spread of misinformation and disinformation on social media platforms. |
| Approach: | They investigate the presence and nature of bias within large language models and its consequential impact on media bias detection. |
| Outcome: | The proposed debiasing strategies include prompt engineering and model fine-tuning. |
McBE: A Multi-task Chinese Bias Evaluation Benchmark for Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing datasets on bias evaluation for large language models focus on English and North American culture and are limited to one task. |
| Approach: | They propose to evaluate Chinese language models' biases from multiple perspectives using a multi-task Chinese Bias Evaluation Benchmark. |
| Outcome: | The proposed model covers 12, 82 subcategories and 5 evaluation tasks covering a wide range of categories and content diversity. |
From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models (2023.acl-long)
Copied to clipboard
| Challenge: | Hundreds of studies have highlighted ethical issues in NLP models . |
| Approach: | They propose to measure media biases in LMs trained on diverse data sources . they focus on hate speech and misinformation detection . |
| Outcome: | The proposed methods quantify the fairness of downstream NLP models trained on politically biased LMs. |
Stereotype Detection as a Catalyst for Enhanced Bias Detection: A Multi-Task Learning Approach (2025.findings-acl)
Copied to clipboard
| Challenge: | a new study addresses bias and stereotypes in language models by exploring how learning them together improves performance. |
| Approach: | They propose a dataset for bias and stereotype detection that integrates religion, gender, socio-economic status, race, profession, and others. |
| Outcome: | The proposed dataset compares encoder-only models and fine-tuned decoder- only models . the results show that learning stereotypes together improves bias detection . |
Multi-Modal Bias: Introducing a Framework for Stereotypical Bias Assessment beyond Gender and Race in Vision–Language Models (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent advances in self-supervised training have led to a new class of pretrained vision–language models. |
| Approach: | They propose a visual and textual bias benchmark to assess bias in self-supervised multimodal models using 3,800 images and phrases from 14 population subgroups. |
| Outcome: | The proposed model shows that it favors certain groups while maintaining the accuracy of the model. |
Detecting and Reducing Bias in a High Stakes Domain (D19-1)
Copied to clipboard
| Challenge: | Existing research shows that a deep learning model can predict aggression and loss in posts by focusing on stop words such as “a” or “on”. |
| Approach: | They developed an approach to interpret a deep learning model that often bases its predictions on stop words such as "a" or "on" to tackle bias, they annotated the rationales and built models that drastically reduce bias. |
| Outcome: | The proposed model can predict aggression and loss in posts by using stop words such as "a" or "on" the new annotations enable us to quantitatively measure how justified the model predictions are, and build models that drastically reduce bias. |