| Challenge: | amplification is a phenomenon in which models exacerbate biases or stereotypes in training data. |
| Approach: | They compare gender ratios in training vs. generated images to investigate bias amplification . they find that a model amplifys gender-occupation biases considerably . |
| Outcome: | The proposed model amplifys gender-occupation biases in training data, but it can be attributed to discrepancies between training captions and model prompts. |
Similar Papers
Mitigating Gender Bias Amplification in Distribution by Posterior Regularization (2020.acl-main)
Copied to clipboard
| Challenge: | Recent studies show that data-driven machine learning models carry societal biases in the dataset they trained on. |
| Approach: | They propose to calibrate top predictions of a model by injecting corpus-level constraints to ensure that the gender disparity is not amplified. |
| Outcome: | The proposed method can almost remove bias amplification in the distribution with little loss of performance. |
Evaluating Bias and Fairness in Gender-Neutral Pretrained Vision-and-Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Pretrained machine learning models perpetuate and even amplify existing biases in data . this can result in unfair outcomes that ultimately impact user experience . |
| Approach: | They quantify bias amplification in pretraining and after fine-tuning on vision-and-language models. |
| Outcome: | The results show that pretrained models can perpetuate and even amplify biases in data without compromising performance. |
T2IAT: Measuring Valence and Stereotypical Biases in Text-to-Image Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in text-to-image generative models have produced high quality images with a breakthrough of inference speed. |
| Approach: | They propose a text-to-image association test framework that quantifies implicit stereotypes between concepts and valence and those in images. |
| Outcome: | The proposed framework quantifies implicit stereotypes between concepts and valence and those in images. |
A Prompt Array Keeps the Bias Away: Debiasing Vision-Language Models with Adversarial Learning (2022.aacl-main)
Copied to clipboard
| Challenge: | Large-scale, pretrained vision-language models are growing in popularity due to impressive performance on downstream tasks with minimal finetuning. |
| Approach: | They propose to apply ranking metrics to image-text representations to investigate bias measures and debiasing methods to reduce various bias measures. |
| Outcome: | The proposed model reduces bias measures with minimal degradation to image-text representations. |
How well can Text-to-Image Generative Models understand Ethical Natural Language Interventions? (2022.emnlp-main)
Copied to clipboard
| Challenge: | Text-to-image generative models can generate high-quality photo-realistic images conditional on natural language text descriptions in a zero-shot fashion. |
| Approach: | They propose an Ethical NaTural Language Interventions in Text-to-Image GENeration benchmark dataset to evaluate the change in image generation conditional on ethical interventions across three social axes – gender, skin color, and culture. |
| Outcome: | The proposed model generations cover diverse social groups while preserving image quality. |
Understanding and Mitigating Bias Inheritance in LLM-based Data Augmentation on Downstream Tasks (2026.acl-long)
Copied to clipboard
Miaomiao Li, Hao Chen, Yang Wang, Tingyuan Zhu, Weijia Zhang, Kaijie Zhu, Kam-Fai Wong, Jindong Wang
| Challenge: | Generating synthetic datasets via large language models (LLMs) has emerged as promising approach to improve LLM performance. |
| Approach: | They propose three mitigation strategies to mitigate bias inheritance in LLMs by analyzing real and LLM-augmented data. |
| Outcome: | The proposed methods can work differently on different tasks and biases. |
Precision or Recall? An Analysis of Image Captions for Training Text-to-Image Generation Model (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in text-to-image models have demonstrated remarkable capabilities in image synthesis. |
| Approach: | They analyze the critical role of caption precision and recall in text-to-image model training. |
| Outcome: | The proposed model trains with synthetic captions that show similar behavior to those trained on human-annotated captions. |
Exposure Bias versus Self-Recovery: Are Distortions Really Incremental for Autoregressive Text Generation? (2021.emnlp-main)
Copied to clipboard
| Challenge: | Exposure bias is a central problem for auto-regressive language models (LM) it is believed that teacher forcing would cause test-time generation to be incrementally distorted due to the training-generation discrepancy. |
| Approach: | They propose to quantify the impact of exposure bias in quality, diversity, consistency and consistency by using ground-truth data prefixes instead of prefix generated by the model. |
| Outcome: | The proposed model performs better when the training-generation discrepancy is removed . the model is more robust and self-recovery ability is shown to counter exposure bias. |
Are Gender-Neutral Queries Really Gender-Neutral? Mitigating Gender Bias in Image Search (2021.emnlp-main)
Copied to clipboard
| Challenge: | Extensive experiments on MS-COCO and Flickr30K benchmarks show that our methods significantly reduce the gender bias in image search models. |
| Approach: | They propose a fair sampling method and a feature clipping method to debias image search models. |
| Outcome: | The proposed methods significantly reduce gender bias in image search models on MS-COCO and Flickr30K benchmarks. |
Intrinsic Bias is Predicted by Pretraining Data and Correlates with Downstream Performance in Vision-Language Encoders (2025.naacl-long)
Copied to clipboard
| Challenge: | Recent work has found that vision-language models trained under the Contrastive Language Image Pre-training framework contain intrinsic social biases, but how these biase relates to downstream performance has been unclear. |
| Approach: | They present the largest comprehensive analysis to-date of how upstream pre-training factors and downstream performance of CLIP models relate to their intrinsic biases. |
| Outcome: | The proposed model performance analysis shows that the choice of pre-training dataset is the most significant upstream predictor of bias, whereas architectural variations have minimal impact. |