One Sentence, Two Embeddings: Contrastive Learning of Explicit and Implicit Semantic Representations (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing sentence embedding methods lack the ability to capture the implicit semantics of sentences. |
| Approach: | They propose a sentence embedding method that assigns two embeddables to each sentence . one represents the explicit semantics and the other represents the implicit semantics . results show DualCSE can effectively encode both explicit and implicit meanings - they argue . |
| Outcome: | The proposed method can effectively encode both explicit and implicit meanings and improve the performance of the downstream task. |
Similar Papers
SimCSE: Simple Contrastive Learning of Sentence Embeddings (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for learning universal sentence embeddings are based on unsupervised approaches with only dropout as noise. |
| Approach: | They propose an unsupervised approach that takes an input sentence and predicts itself in a contrastive objective with only standard dropout used as noise. |
| Outcome: | The proposed framework performs on par with previous supervised approaches and can produce superior sentence embeddings from unlabeled or labeled data. |
Contrasting distinct structured views to learn sentence embeddings (2021.eacl-srw)
Copied to clipboard
| Challenge: | Existing methods to build sentence embeddings rely on a similar Recurrent Neural Network (RNN) heterogeneity of performances across models and tasks makes us assume some structures might be better adapted given the considered task or sentence. |
| Approach: | They propose a self-supervised method that builds sentence embeddings from syntactic structures . they hypothesize that some linguistic representations might be better adapted given the task . |
| Outcome: | The proposed method outperforms comparable methods on several tasks from standard sentence embedding benchmarks. |
Sentence Representations via Gaussian Embedding (2024.eacl-short)
Copied to clipboard
| Challenge: | Sentence embeddings represent a sentence's meaning as a point in a vector space and primarily use symmetric measures such as the cosine similarity to measure the similarity between sentences, they cannot capture asymmetric relationships between two sentences, such as entailment and hierarchical relations. |
| Approach: | They propose a Gaussian-distribution-based contrastive learning framework for sentence embedding that can handle asymmetric inter-sentential relations and a similarity measure for identifying entailment relations. |
| Outcome: | The proposed framework performs comparable to that of previous methods on natural language inference tasks and estimates direction of entailment relations, which is difficult with point representations. |
InfoCSE: Information-aggregated Contrastive Learning of Sentence Embeddings (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies on contrastive learning for sentence embeddings are weak . researchers have started to use contrastive training to learn better unsupervised sentences. |
| Approach: | They propose an information-aggregated contrastive learning framework for learning unsupervised sentence embeddings. |
| Outcome: | The proposed framework outperforms SimCSE on several benchmark datasets w.r.t the semantic text similarity task. |
Static Word Embeddings for Sentence Semantic Representation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to learn fixed-length embeddings for sentence semantics require large computational cost, making it difficult to process billions of sentences cost-efficiently or deploy models on resource-constrained devices such as smartphones. |
| Approach: | They propose to extract word embeddings from a pre-trained Sentence Transformer and improve them with sentence-level principal component analysis followed by knowledge distillation or contrastive learning. |
| Outcome: | The proposed model outperforms existing models on sentence semantic tasks and surpasses a basic Sentence Transformer model (SimCSE) on a text embedding benchmark. |
AugCSE: Contrastive Sentence Embedding with Diverse Augmentations (2022.aacl-main)
Copied to clipboard
| Challenge: | Similar work has shown that a single augmentation can be used to learn a robust generalpurpose representation with contrastive learning. |
| Approach: | They propose a unified framework to utilize diverse sets of data augmentations to achieve a better, general-purpose sentence embedding model. |
| Outcome: | The proposed framework achieves state-of-the-art results on downstream transfer tasks and performs competitively on semantic textual similarity tasks, using only unsupervised data. |
Cross-lingual Sentence Embedding using Multi-Task Learning (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing multilingual sentence embedding models require large parallel corpora to learn efficiently, limiting their scope. |
| Approach: | They propose a sentence embedding framework based on an unsupervised loss function . they capture semantic similarity and relatedness between sentences using a multi-task loss function. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on STS, BUCC and Tatoeba benchmarks and on a monolingual benchmark. |
Aligning Cross-lingual Sentence Representations with Dual Momentum Contrast (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing work uses sentences within the same batch as negatives, which suffers from easy negatives. |
| Approach: | They propose to align sentence representations from different languages into a unified embedding space . they adapt MoCo to further improve the quality of alignment . |
| Outcome: | The proposed model achieves state-of-the-art on several tasks. |
What you can cram into a single $&!#* vector: Probing sentence embeddings for linguistic properties (P18-1)
Copied to clipboard
| Challenge: | a lack of understanding of the properties of sentence embeddings is limiting the use of the techniques. |
| Approach: | They propose 10 probing tasks designed to capture simple linguistic features of sentences . they use three different encoders to train embeddings in eight different ways . |
| Outcome: | The proposed tasks capture key linguistic features of sentences, but they are difficult to infer from them. |
Sub-Sentence Encoder: Contrastive Learning of Propositional Semantic Representations (2024.naacl-long)
Copied to clipboard
Sihao Chen, Hongming Zhang, Tong Chen, Ben Zhou, Wenhao Yu, Dian Yu, Baolin Peng, Hongwei Wang, Dan Roth, Dong Yu
| Challenge: | Sentence embeddings are typically learned to recognize the semantic relation between two text inputs. |
| Approach: | They introduce a contrastively-learned contextual embedding model for fine-grained semantic representation of text. |
| Outcome: | The proposed model is able to produce contextual embeddings corresponding to different atomic propositions, i.e. semantic equivalence between propositions across different text sequences. |