Encouraging Paragraph Embeddings to Remember Sentence Identity Improves Classification (P19-1)
Copied to clipboard
| Challenge: | Existing paragraph embedding methods do not capture basic linguistic properties, but their performance is limited. |
| Approach: | They propose a paragraph embedding method that can't tell whether a sentence occurs in a given paragraph. |
| Outcome: | The proposed method outperforms reconstruction-based methods on a semi-supervised dataset and improves on benchmark datasets. |
Similar Papers
What you can cram into a single $&!#* vector: Probing sentence embeddings for linguistic properties (P18-1)
Copied to clipboard
| Challenge: | a lack of understanding of the properties of sentence embeddings is limiting the use of the techniques. |
| Approach: | They propose 10 probing tasks designed to capture simple linguistic features of sentences . they use three different encoders to train embeddings in eight different ways . |
| Outcome: | The proposed tasks capture key linguistic features of sentences, but they are difficult to infer from them. |
More Discriminative Sentence Embeddings via Semantic Graph Smoothing (2024.eacl-short)
Copied to clipboard
| Challenge: | Text categorization is a natural language processing task that involves arranging texts into coherent groups based on their content. |
| Approach: | They propose to use semantic graph smoothing to enhance sentence embeddings from pretrained models to improve results for supervised and unsupervised document categorization tasks. |
| Outcome: | The proposed method improves sentences embeddings for supervised and unsupervised document categorization tasks. |
Vec2Sent: Probing Sentence Embeddings with Natural Language Generation (2020.coling-main)
Copied to clipboard
| Challenge: | a new unsupervised probing task is able to retrieve black-box sentence embeddings . a variety of problems surround probing tasks, including manual construing . |
| Approach: | They propose a method to generate black-box sentence embeddings by conditionally generating from them . they also illustrate how the language generated from different encoders differs . |
| Outcome: | The proposed probing task improves the performance of black-box sentence embeddings . the proposed task is based on a conditional natural language generation approach . |
Are the Best Multilingual Document Embeddings simply Based on Sentence Embeddings? (2023.findings-eacl)
Copied to clipboard
| Challenge: | obtaining document embeddings at document level is challenging due to computational requirements and lack of appropriate data. |
| Approach: | They compare methods to produce document-level representations from sentences based on LASER, LaBSE, and Sentence BERT pre-trained multilingual models. |
| Outcome: | The proposed methods produce document-level representations from sentences in 8 languages . the results show that a clever combination of sentence embeddings is usually better than encoding the full document as a single unit. |
Static Word Embeddings for Sentence Semantic Representation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to learn fixed-length embeddings for sentence semantics require large computational cost, making it difficult to process billions of sentences cost-efficiently or deploy models on resource-constrained devices such as smartphones. |
| Approach: | They propose to extract word embeddings from a pre-trained Sentence Transformer and improve them with sentence-level principal component analysis followed by knowledge distillation or contrastive learning. |
| Outcome: | The proposed model outperforms existing models on sentence semantic tasks and surpasses a basic Sentence Transformer model (SimCSE) on a text embedding benchmark. |
Empirical Linguistic Study of Sentence Embeddings (P19-1)
Copied to clipboard
| Challenge: | a new method of analysing sentence embeddings shows that linguistic information is retained in the vector representations of sentences. |
| Approach: | They propose a method of analysing the content of sentence embeddings based on probing tasks and contrasting languages. |
| Outcome: | The proposed method is based on probing tasks and classification datasets for two contrasting languages. |
Exploring Semantic Properties of Sentence Embeddings (P18-2)
Copied to clipboard
| Challenge: | Neural vector representations are ubiquitous throughout all subfields of natural language processing. |
| Approach: | They propose a framework that generates triplets of sentences to explore how changes in the syntactic structure or semantics of a given sentence affect their similarity. |
| Outcome: | The proposed framework generates triplets of sentences to explore how changes in the syntactic structure or semantics of a given sentence affect the similarities obtained between their embeddings. |
Text Embeddings Reveal (Almost) As Much As Text (2023.emnlp-main)
Copied to clipboard
| Challenge: | a vector database of dense text embeddings stores only the text data, not the original text . a multi-step method that iteratively corrects and re-embeds text can recover 92% of 32-token text inputs exactly. |
| Approach: | They propose a method that iteratively corrects and re-embeds text to recover 92% of 32-token text inputs exactly. |
| Outcome: | The proposed method recovers 92% of 32-token text inputs exactly. |
Sentence Embedding Leaks More Information than You Expect: Generative Embedding Inversion Attack to Recover the Whole Sentence (2023.findings-acl)
Copied to clipboard
| Challenge: | Sentence-level representations are beneficial for various natural language processing tasks. |
| Approach: | They propose a generative embedding inversion attack that reconstructs input sequences based only on their sentence embeddeds. |
| Outcome: | The proposed model outperforms previous embedding inversion attacks in classification metrics and generates coherent and contextually similar sentences as original inputs. |
Semantic Geometry of Sentence Embeddings (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Sentence embeddings are central to natural language processing, but their internal features are not interpretable and users lack fine-grained control for downstream tasks. |
| Approach: | They propose a formal framework to characterize the organization of features in sentence embeddings . they show how they can be composed to capture richer semantic structures . |
| Outcome: | The proposed method can be used to capture richer semantic structures. |