Misalignment Attack on Text-to-Image Models via Text Embedding Optimization and Inversion (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Text embedding is a key component of modern NLP models but also poses additional risks. |
| Approach: | They propose a framework that optimizes embeddings and inverts them to obtain misaligned prompts. |
| Outcome: | The proposed framework exploits the continuity and distribution characteristics of text embeddings to obtain misaligned prompts of discrete tokens. |
Similar Papers
Uncovering Limitations in Text-to-Image Generation: A Contrastive Approach with Structured Semantic Alignment (2023.findings-emnlp)
Copied to clipboard
| Challenge: | a new method for text-to-image generation models is proposed to address these limitations . SSA focuses on learning structured semantic embeddings across different modalities . |
| Approach: | They propose a method to evaluate text-to-image generation models using structured semantic embeddings . they propose to learn mutated prompts by substituting words with equivalent or nonequivalent alternatives . |
| Outcome: | The proposed method improves the measurement of semantic consistency of text-to-image generation models. |
Asymmetric Bias in Text-to-Image Generation with Adversarial Attacks (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on adversarial attacks against text-to-image models have not explored the reasons for their effectiveness. |
| Approach: | They propose an adversarial attack objective - entity swapping using adversarials and two gradient-based attack algorithms. |
| Outcome: | The proposed attack objective - entity swapping - is easier to replace “human” with “robot” in the prompt “a human dancing in the rain.” |
Adversarial Attacks on Parts of Speech: An Empirical Study in Text-to-Image Generation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies show that text-to-image models are vulnerable to adversarial perturbations . |
| Approach: | They investigate the impact of adversarial attacks on different POS tags within text prompts on T2I models. |
| Outcome: | The proposed model is vulnerable to adversarial perturbations with noun perturbations in text prompts. |
Transferable Embedding Inversion Attack: Uncovering Privacy Risks in Text Embeddings without Model Queries (2024.acl-long)
Copied to clipboard
| Challenge: | Recent advances in text embedding models have significantly streamlined the process of generating embeddables. |
| Approach: | They develop a transfer attack method that uses a surrogate model to mimic the victim model's behavior and infers sensitive information from embeddings without direct access. |
| Outcome: | The proposed method outperforms existing methods and reveals potential privacy vulnerabilities in embedding technologies. |
ALGEN: Few-shot Inversion Attacks on Textual Embeddings via Cross-Model Alignment and Generation (2025.acl-long)
Copied to clipboard
| Challenge: | Recent studies have proven that private textual data is vulnerable to inversion attacks . authors present a method to reduce the cost of inversion and improve performance . |
| Approach: | They propose a method that aligns victim embeddings to attack space and reconstructs text . they find that none of the defense mechanisms are effective against inversion attacks . |
| Outcome: | The proposed method lowers the cost of inversion and improves performance across languages and domains. |
Jailbreaking Prompt Attack: A Controllable Adversarial Attack against Diffusion Models (2025.findings-naacl)
Copied to clipboard
| Challenge: | Text-to-image (T2I) models can be used to generate harmful content such as sexually explicit, unfaithful, and misleading or Not-Safe-for-Work (NSFW) images. |
| Approach: | They propose a more practical and universal attack that does not require the presence of a target model. |
| Outcome: | The proposed attack bypasses both text and image safety checkers while preserving high semantic alignment with the target prompt. |
Multimodal Pragmatic Jailbreak on Text-to-image Models (2025.acl-long)
Copied to clipboard
Tong Liu, Zhixin Lai, Jiawen Wang, Gengyuan Zhang, Shuo Chen, Philip Torr, Vera Demberg, Volker Tresp, Jindong Gu
| Challenge: | Existing jailbreaks for diffusion-based text-to-image models generate unsafe content . experimental results show that all tested models suffer from unsafe generation . |
| Approach: | They propose a jailbreak that triggers diffusion-based text-to-image models to generate the image with visual text, resulting in unsafe content. |
| Outcome: | The proposed model generates image with visual text, but the model is unsafe under such jailbreak. |
A Strong and Robust Baseline for Text-Image Matching (P19-2)
Copied to clipboard
| Challenge: | Text-image matching is one of the most popular methods for training text-image embeddings. |
| Approach: | They propose to use a kNN-margin loss that utilizes hard negatives and is robust to noise . they advocate using Inverted Softmax and Cross-modal Local Scaling during inference . |
| Outcome: | The proposed loss function is robust to noise and pseudo negatives are tolerable . the proposed loss functions improve scores of all metrics by a large margin . |
T3: Tree-Autoencoder Constrained Adversarial Text Generation for Targeted Attack (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing adversarial examples can induce arbitrary errors to the target models, but they can be exploited to estimate robustness of NLP models. |
| Approach: | They propose a target-controllable adversarial attack framework T3 to handle adversarials . they use tree-based decoders to regularize the syntactic correctness of generated text . |
| Outcome: | The proposed framework can be used to estimate the robustness of NLP models. |
Multilingual Text-to-Image Generation Magnifies Gender Stereotypes (2025.acl-long)
Copied to clipboard
Felix Friedrich, Katharina Hämmerl, Patrick Schramowski, Manuel Brack, Jindřich Libovický, Alexander Fraser, Kristian Kersting
| Challenge: | Text-to-image (T2I) generation models have great results in image quality, flexibility, and text alignment, but they suffer from substantial gender bias. |
| Approach: | They propose a benchmark to study gender bias in multilingual T2I models . they use multilingual prompts to account for grammatical differences influencing gender . |
| Outcome: | The proposed benchmark shows strong gender biases and language-specific differences across models. |