Sentence Embedding Leaks More Information than You Expect: Generative Embedding Inversion Attack to Recover the Whole Sentence (2023.findings-acl)
Copied to clipboard
| Challenge: | Sentence-level representations are beneficial for various natural language processing tasks. |
| Approach: | They propose a generative embedding inversion attack that reconstructs input sequences based only on their sentence embeddeds. |
| Outcome: | The proposed model outperforms previous embedding inversion attacks in classification metrics and generates coherent and contextually similar sentences as original inputs. |
Similar Papers
ALGEN: Few-shot Inversion Attacks on Textual Embeddings via Cross-Model Alignment and Generation (2025.acl-long)
Copied to clipboard
| Challenge: | Recent studies have proven that private textual data is vulnerable to inversion attacks . authors present a method to reduce the cost of inversion and improve performance . |
| Approach: | They propose a method that aligns victim embeddings to attack space and reconstructs text . they find that none of the defense mechanisms are effective against inversion attacks . |
| Outcome: | The proposed method lowers the cost of inversion and improves performance across languages and domains. |
Text Embeddings Reveal (Almost) As Much As Text (2023.emnlp-main)
Copied to clipboard
| Challenge: | a vector database of dense text embeddings stores only the text data, not the original text . a multi-step method that iteratively corrects and re-embeds text can recover 92% of 32-token text inputs exactly. |
| Approach: | They propose a method that iteratively corrects and re-embeds text to recover 92% of 32-token text inputs exactly. |
| Outcome: | The proposed method recovers 92% of 32-token text inputs exactly. |
Are the Best Multilingual Document Embeddings simply Based on Sentence Embeddings? (2023.findings-eacl)
Copied to clipboard
| Challenge: | obtaining document embeddings at document level is challenging due to computational requirements and lack of appropriate data. |
| Approach: | They compare methods to produce document-level representations from sentences based on LASER, LaBSE, and Sentence BERT pre-trained multilingual models. |
| Outcome: | The proposed methods produce document-level representations from sentences in 8 languages . the results show that a clever combination of sentence embeddings is usually better than encoding the full document as a single unit. |
Text Embedding Inversion Security for Multilingual Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | storing sensitive information as embeddings is susceptible to security breaches, as text can be reconstructed from embeddables . study explores multilingual inversion attacks using a masking defense . |
| Approach: | They propose a simple masking defense that can be used to decode embedded text . they define the problem of black-box multilingual and crosslingual inversion attacks . |
| Outcome: | The proposed defense is effective for both monolingual and multilingual models. |
GASE: Generatively Augmented Sentence Encoding (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Generatively Augmented Sentence Encoding variates the input text by paraphrasing, summarizing, or extracting keywords, followed by pooling the original and synthetic embeddings. |
| Approach: | They propose a training-free approach to improve sentence embeddings by applying generative text models for data augmentation at inference time. |
| Outcome: | The proposed approach does not require access to model parameters or computational resources typically required for fine-tuning state-of-the-art models. |
What you can cram into a single $&!#* vector: Probing sentence embeddings for linguistic properties (P18-1)
Copied to clipboard
| Challenge: | a lack of understanding of the properties of sentence embeddings is limiting the use of the techniques. |
| Approach: | They propose 10 probing tasks designed to capture simple linguistic features of sentences . they use three different encoders to train embeddings in eight different ways . |
| Outcome: | The proposed tasks capture key linguistic features of sentences, but they are difficult to infer from them. |
Exploiting Invertible Decoders for Unsupervised Sentence Representation Learning (P19-1)
Copied to clipboard
| Challenge: | Encoder-decoder models for unsupervised sentence representation learning discard decoder after training . decoded sentences are often used to make better predictions of words in a given sentence . |
| Approach: | They propose two types of decoding functions whose inverse can be easily derived without expensive inverse calculation. |
| Outcome: | The proposed models can learn good representations from encoders and decoders without expensive calculations. |
ObfusLM: Privacy-preserving Language Model Service against Embedding Inversion Attacks (2025.acl-long)
Copied to clipboard
Yu Lin, Ruining Yang, Yunlong Mao, Qizhi Zhang, Jue Hong, Quanwei Cai, Ye Wu, Huiqi Liu, Zhiyu Chen, Bing Duan, Sheng Zhong
| Challenge: | Recent studies show that obfuscation techniques for MLaaS are susceptible to embedding inversion attacks (EIAs). |
| Approach: | They propose a model obfuscation framework that protects client inputs from embedding inversion attacks by obliviously obbing models. |
| Outcome: | The proposed framework outperforms existing works in utility by 10% with a nearly 80% resistance rate against embedding inversion attacks. |
On the Sentence Embeddings from Pre-trained Language Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained contextual representations like BERT have been widely used for NLP tasks. |
| Approach: | They propose to transform anisotropic sentence embedding distribution to smooth and isotropic Gaussian distribution by normalizing flows that are learned with an unsupervised objective. |
| Outcome: | The proposed method achieves significant performance gains over state-of-the-art embeddings on a variety of semantic textual similarity tasks. |
Vec2Sent: Probing Sentence Embeddings with Natural Language Generation (2020.coling-main)
Copied to clipboard
| Challenge: | a new unsupervised probing task is able to retrieve black-box sentence embeddings . a variety of problems surround probing tasks, including manual construing . |
| Approach: | They propose a method to generate black-box sentence embeddings by conditionally generating from them . they also illustrate how the language generated from different encoders differs . |
| Outcome: | The proposed probing task improves the performance of black-box sentence embeddings . the proposed task is based on a conditional natural language generation approach . |