| Challenge: | Existing studies based on image checker, model fine-tuning and embedding blocking are impractical in real-world applications. |
| Approach: | They propose a novel reward function measuring toxicity and text alignment of generated images and train the optimizer through Proximal Policy Optimization. |
| Outcome: | The proposed model reduces the likelihood of various models in generating inappropriate images, with no significant impact on text alignment. |
Similar Papers
Iterative Prompt Refinement for Safer Text-to-Image Generation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing safety methods for text-to-image models ignore the images produced . this can result in unsafe outputs or unnecessary changes to already safe prompts . |
| Approach: | They propose an iterative prompt refinement algorithm that uses Vision Language Models to analyze prompts and generated images. |
| Outcome: | The proposed method improves safety while maintaining user intent and reliability comparable to existing methods. |
Jailbreaking Prompt Attack: A Controllable Adversarial Attack against Diffusion Models (2025.findings-naacl)
Copied to clipboard
| Challenge: | Text-to-image (T2I) models can be used to generate harmful content such as sexually explicit, unfaithful, and misleading or Not-Safe-for-Work (NSFW) images. |
| Approach: | They propose a more practical and universal attack that does not require the presence of a target model. |
| Outcome: | The proposed attack bypasses both text and image safety checkers while preserving high semantic alignment with the target prompt. |
Multimodal Pragmatic Jailbreak on Text-to-image Models (2025.acl-long)
Copied to clipboard
Tong Liu, Zhixin Lai, Jiawen Wang, Gengyuan Zhang, Shuo Chen, Philip Torr, Vera Demberg, Volker Tresp, Jindong Gu
| Challenge: | Existing jailbreaks for diffusion-based text-to-image models generate unsafe content . experimental results show that all tested models suffer from unsafe generation . |
| Approach: | They propose a jailbreak that triggers diffusion-based text-to-image models to generate the image with visual text, resulting in unsafe content. |
| Outcome: | The proposed model generates image with visual text, but the model is unsafe under such jailbreak. |
Prompt Expansion for Adaptive Text-to-Image Generation (2024.acl-long)
Copied to clipboard
| Challenge: | Text-to-image generation models are powerful but difficult to use. Users craft specific prompts to get better images, though the prompts can be repetitive. |
| Approach: | They propose a framework that takes a text query as input and outputs a set of expanded text prompts that are optimized to generate a wider variety of appealing images. |
| Outcome: | The proposed framework generates high-quality images from text prompts with less effort and is more aesthetically pleasing than baseline models. |
BeautifulPrompt: Towards Automatic Prompt Engineering for Text-to-Image Synthesis (2023.emnlp-industry)
Copied to clipboard
| Challenge: | Recent text-to-image models require multiple passes of prompt engineering by humans to produce satisfactory results for real-world applications. |
| Approach: | They propose a deep generative model to generate high-quality prompts from raw descriptions using visual feedback. |
| Outcome: | The proposed model produces high-quality prompts from simple raw descriptions . it can be integrated to a cloud-native AI platform to provide better image generation service in the cloud. |
Collaborative Generative AI: Integrating GPT-k for Efficient Editing in Text-to-Image Generation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results show that GPT-k models focus more on inserting modifiers than predicting spontaneous changes in the primary subject matter. |
| Approach: | They compare the common edits made by humans and GPT-k models to examine their performance in prompting T2I. |
| Outcome: | The proposed models improve the prompt editing process by 20-30%, the authors show . they show that humans tend to replace words and phrases with modifiers . |
NeuroPrompts: An Adaptive Framework to Optimize Prompts for Text-to-Image Generation (2024.eacl-demo)
Copied to clipboard
| Challenge: | Recent advances in text-to-image diffusion models have made it difficult to obtain high-quality images. |
| Approach: | They propose an adaptive framework that automatically enhances a user's prompt to improve the quality of generation models. |
| Outcome: | The proposed framework generates prompts similar to those produced by human prompt engineers and provides user control over stylistic features via constraint set specification. |
Prompt Refinement with Image Pivot for Text-to-Image Generation (2024.acl-long)
Copied to clipboard
| Challenge: | Recent advances in text-to-image generation have markedly expanded the boundaries of digital artistry, enabling the creation of visually compelling images with unprecedented ease. |
| Approach: | They propose to decompose the prompt refinement process into two tasks: inferring user-preferred images from user languages and translating them into system languages. |
| Outcome: | Experiments show that PRIP outperforms baselines and transfers to unseen systems in a zero-shot manner. |
Self-Rewarding Large Vision-Language Models for Optimizing Prompts in Text-to-Image Generation (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for rewriting text-to-image models require specialized vocabulary . a new approach uses large vision language models to optimize text-based models . |
| Approach: | They propose a prompt optimization framework that rephrases a user prompt into a text-to-image model by using large vision language models as solver and reward model. |
| Outcome: | The proposed model outperforms existing models on two popular datasets. |
Jailbreaking Safeguarded Text-to-Image Models via Large Language Models (2026.findings-eacl)
Copied to clipboard
| Challenge: | Text-to-image models generate harmful content when unsafe prompts are submitted . authors propose a method to jailbreak text-to image models with safety guardrails . |
| Approach: | They propose a method to jailbreak text-to-image models with safety guardrails . they use a fine-tuned large language model to generate adversarial prompts based on unsafe prompts. |
| Outcome: | The proposed method bypasses safety guardrails and outperforms existing no-box attacks . the proposed method generates adversarial prompts efficiently after fine-tuning the model . |