The Factuality Tax of Diversity-Intervened Text-to-Image Generation: Benchmark and Fact-Augmented Intervention (2024.emnlp-main)
Copied to clipboard
| Challenge: | Prompt-based “diversity interventions” are commonly adopted to improve the diversity of Text-to-Image models depicting individuals with diverse racial or gender traits. |
| Approach: | They propose a benchmark to quantify the trade-off between using diversity interventions and preserving demographic factuality in T2I models. |
| Outcome: | The proposed model significantly improves the demographic factuality under diversity interventions while preserving diversity. |
Similar Papers
T2I-FactualBench: Benchmarking the Factuality of Text-to-Image Models with Knowledge-Intensive Concepts (2025.acl-long)
Copied to clipboard
Ziwei Huang, Wanggui He, Quanyu Long, Yandi Wang, Haoyuan Li, Zhelun Yu, Fangxun Shu, Weilong Dai, Hao Jiang, Fei Wu, Leilei Gan
| Challenge: | Existing studies on text-to-image (T2I) models focus on text alignment, image quality, and object composition capabilities. |
| Approach: | They propose a T2I-FactualBench benchmark to evaluate the factuality of knowledge-intensive concept generation. |
| Outcome: | The proposed framework evaluates the factuality of knowledge-intensive concept generation tasks. |
How well can Text-to-Image Generative Models understand Ethical Natural Language Interventions? (2022.emnlp-main)
Copied to clipboard
| Challenge: | Text-to-image generative models can generate high-quality photo-realistic images conditional on natural language text descriptions in a zero-shot fashion. |
| Approach: | They propose an Ethical NaTural Language Interventions in Text-to-Image GENeration benchmark dataset to evaluate the change in image generation conditional on ethical interventions across three social axes – gender, skin color, and culture. |
| Outcome: | The proposed model generations cover diverse social groups while preserving image quality. |
The Male CEO and the Female Assistant: Evaluation and Mitigation of Gender Biases in Text-To-Image Generation of Dual Subjects (2025.acl-long)
Copied to clipboard
| Challenge: | Recent large-scale T2I models like DALLE-3 have made progress in reducing gender stereotypes when generating single-person images. |
| Approach: | They propose a framework that queries T2I models to depict two individuals with gender-stereotyped social identities to evaluate gender biases. |
| Outcome: | The proposed framework reduces gender stereotypes when generating images with more than one person. |
Who Gets Which Message? Auditing Demographic Bias in LLM-Generated Targeted Text (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models generate demographically conditioned persuasive texts at scale . authors argue that such capabilities raise questions about fairness and representational bias in automated communication. |
| Approach: | They propose a framework for evaluating demographic-conditioned targeted messages . they find gender- and age-based asymmetries in male- and youth-targeted messages a . |
| Outcome: | The proposed framework evaluates generated messages across three dimensions: lexical content, language style, and persuasive framing. |
FactVerse: A Benchmark for Factual Consistency in Interleaved Image–Text Generation (2026.acl-long)
Copied to clipboard
Yubo Shan, Kun Zhang, Qiming Xu, Liping Cao, Yingying Cao, Jian Zhang, Yu Wang, Jingyuan Li, Yuanzhuo Wang
| Challenge: | Existing benchmarks lack effective mechanisms to evaluate factual consistency in interleaved image-text generation. |
| Approach: | They propose a benchmark dedicated to evaluating factual consistency in interleaved image-text generation. |
| Outcome: | The proposed framework outperforms existing evaluation methods in evaluating factual consistency in interleaved image-text generation. |
Merging Facts, Crafting Fallacies: Evaluating the Contradictory Nature of Aggregated Factual Claims in Long-Form Generations (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing factuality metrics cannot evaluate paragraphs with ambiguous entities, authors show . |
| Approach: | They propose a new metric to evaluate the factuality of long-form generations from large language models. |
| Outcome: | The proposed metric can assess the factuality of people biographies with entity ambiguity better than FActScore. |
Leveraging Prototypical Representations for Mitigating Social Bias without Demographic Information (2024.naacl-short)
Copied to clipboard
| Challenge: | Existing approaches to mitigate social biases require explicit annotation of demographic information for each sample. |
| Approach: | They propose a method that leverages predefined demographic texts and incorporates a regularization term during the fine-tuning process to mitigate bias in language models. |
| Outcome: | The proposed method outperforms debiasing methods with limited demographic-annotated data. |
Multilingual Text-to-Image Generation Magnifies Gender Stereotypes (2025.acl-long)
Copied to clipboard
Felix Friedrich, Katharina Hämmerl, Patrick Schramowski, Manuel Brack, Jindřich Libovický, Alexander Fraser, Kristian Kersting
| Challenge: | Text-to-image (T2I) generation models have great results in image quality, flexibility, and text alignment, but they suffer from substantial gender bias. |
| Approach: | They propose a benchmark to study gender bias in multilingual T2I models . they use multilingual prompts to account for grammatical differences influencing gender . |
| Outcome: | The proposed benchmark shows strong gender biases and language-specific differences across models. |
Provenance: A Light-weight Fact-checker for Retrieval Augmented LLM Generation Output (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Existing methods for fact checking RAG outputs rely on large language models. |
| Approach: | They propose a method that computes a factuality score that can be thresholded to yield a binary decision to check RAG outputs. |
| Outcome: | The proposed method is low latency and low cost at run-time and no need for LLM fine-tuning. |
G2: Guided Generation for Enhanced Output Diversity in LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to enhance output diversity but compromise quality of outputs. |
| Approach: | They propose a training-free plug-and-play method that enhances output diversity while preserving generation quality. |
| Outcome: | The proposed method enhances output diversity while maintaining an optimal balance between diversity and quality. |