An Inversion Attack Against Obfuscated Embedding Matrix in Language Model Inference (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have explored transforming user inputs to obfuscated embedded vectors, so that the data will not be eavesdropped by service providers. |
| Approach: | They propose to transform user inputs to obfuscated embedded vectors so that the data will not be eavesdropped by service providers. |
| Outcome: | The proposed inversion attack can recover user input 100% from the obfuscated vectors without a solid and deliberate security design and analysis . |
Similar Papers
ObfusLM: Privacy-preserving Language Model Service against Embedding Inversion Attacks (2025.acl-long)
Copied to clipboard
Yu Lin, Ruining Yang, Yunlong Mao, Qizhi Zhang, Jue Hong, Quanwei Cai, Ye Wu, Huiqi Liu, Zhiyu Chen, Bing Duan, Sheng Zhong
| Challenge: | Recent studies show that obfuscation techniques for MLaaS are susceptible to embedding inversion attacks (EIAs). |
| Approach: | They propose a model obfuscation framework that protects client inputs from embedding inversion attacks by obliviously obbing models. |
| Outcome: | The proposed framework outperforms existing works in utility by 10% with a nearly 80% resistance rate against embedding inversion attacks. |
Text Embedding Inversion Security for Multilingual Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | storing sensitive information as embeddings is susceptible to security breaches, as text can be reconstructed from embeddables . study explores multilingual inversion attacks using a masking defense . |
| Approach: | They propose a simple masking defense that can be used to decode embedded text . they define the problem of black-box multilingual and crosslingual inversion attacks . |
| Outcome: | The proposed defense is effective for both monolingual and multilingual models. |
Transferable Embedding Inversion Attack: Uncovering Privacy Risks in Text Embeddings without Model Queries (2024.acl-long)
Copied to clipboard
| Challenge: | Recent advances in text embedding models have significantly streamlined the process of generating embeddables. |
| Approach: | They develop a transfer attack method that uses a surrogate model to mimic the victim model's behavior and infers sensitive information from embeddings without direct access. |
| Outcome: | The proposed method outperforms existing methods and reveals potential privacy vulnerabilities in embedding technologies. |
ALGEN: Few-shot Inversion Attacks on Textual Embeddings via Cross-Model Alignment and Generation (2025.acl-long)
Copied to clipboard
| Challenge: | Recent studies have proven that private textual data is vulnerable to inversion attacks . authors present a method to reduce the cost of inversion and improve performance . |
| Approach: | They propose a method that aligns victim embeddings to attack space and reconstructs text . they find that none of the defense mechanisms are effective against inversion attacks . |
| Outcome: | The proposed method lowers the cost of inversion and improves performance across languages and domains. |
Sentence Embedding Leaks More Information than You Expect: Generative Embedding Inversion Attack to Recover the Whole Sentence (2023.findings-acl)
Copied to clipboard
| Challenge: | Sentence-level representations are beneficial for various natural language processing tasks. |
| Approach: | They propose a generative embedding inversion attack that reconstructs input sequences based only on their sentence embeddeds. |
| Outcome: | The proposed model outperforms previous embedding inversion attacks in classification metrics and generates coherent and contextually similar sentences as original inputs. |
User Inference Attacks on Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | a large amount of data written by humans is used to train and fine-tune large language models. |
| Approach: | They propose to infer if a user's data was used to train an LLM by using example-level differential privacy. |
| Outcome: | The proposed attacks are easy to employ and only require black-box access to an LLM and a few samples from the user. |
Invernet: An Inversion Attack Framework to Infer Fine-Tuning Datasets through Word Embeddings (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing word embeddings are data intensive and require large-scale training corpus, sufficient training iterations, and high computational capacity. |
| Approach: | They propose a framework that infers context distributions from a downstream dataset and then uses them to fine-tune the embedding. |
| Outcome: | The proposed framework materializes privacy concern by inferring context distribution in the downstream dataset, which can lead to key information breach. |
Be Careful about Poisoned Word Embeddings: Exploring the Vulnerability of the Embedding Layers in NLP Models (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent studies reveal a security threat to natural language processing models, called the Backdoor Attack. |
| Approach: | They propose to hack a model by modifying one single word embedding vector without sacrificing accuracy on clean samples. |
| Outcome: | The proposed method is more efficient and stealthier on sentiment analysis and sentence-pair classification tasks. |
CAPE: Context-Aware Private Embeddings for Private Language Learning (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to obtain text representations or embeddings with these models encoding personally identifiable information may lead to privacy leaks. |
| Approach: | They propose a novel approach which combines differential privacy and adversarial learning to preserve privacy during training of embeddings. |
| Outcome: | The proposed approach reduces private information leakage by 3% over the current method. |
Stealing Training Data from Large Language Models in Decentralized Training through Activation Inversion Attack (2025.acl-long)
Copied to clipboard
| Challenge: | Decentralized training is a resource-efficient framework to democratize training of large language models. |
| Approach: | They propose an activation inversion attack to exploit privacy leakage from training data . they construct a shadow dataset comprising text labels and corresponding activations . |
| Outcome: | The proposed attack surface is based on a shadow dataset and public datasets . the proposed attack model reconstructs training data from activations in victim decentralized training. |