| Challenge: | Semantic textual similarity (STS) is a fundamental NLP task that measures the semantic similarity between two sentences. |
| Approach: | They propose to use a conditional STS dataset to measure sentences’ similarity conditioned on a certain aspect to reduce the inherent ambiguity posed by the sentences. |
| Outcome: | The proposed method improves the performance over baselines on the C-STS dataset with over 80% F1 score. |
Similar Papers
Annotating Training Data for Conditional Semantic Textual Similarity Measurement using Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Semantic similarity between two sentences depends on the aspects considered between those sentences. |
| Approach: | They propose a Conditional Semantic Textual Similarity task which measures the similarity between two sentences under a specified condition. |
| Outcome: | The proposed method improves Spearman correlation by 5.4% by training a supervised model on the re-annotated dataset. |
C-STS: Conditional Semantic Textual Similarity (2023.emnlp-main)
Copied to clipboard
Ameet Deshpande, Carlos Jimenez, Howard Chen, Vishvak Murahari, Victoria Graf, Tanmay Rajpurohit, Ashwin Kalyan, Danqi Chen, Karthik Narasimhan
| Challenge: | Semantic textual similarity (STS) is a cornerstone task in natural language processing, but it is inherently ambiguous. |
| Approach: | They propose a task called conditional STS which measures similarity conditioned on an aspect elucidated in natural language. |
| Outcome: | The proposed task reduces subjectivity and ambiguity and enables fine-grained similarity evaluation using diverse conditions. |
Conditional Semantic Textual Similarity via Conditional Contrastive Learning (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods to assess similarity between sentences encounter over-estimation problem . compared to fuzzy representations, similarity is comparatively lower in terms of "The person's age". |
| Approach: | They propose a conditional contrastive learning framework that constructs positive and negative samples from two perspectives. |
| Outcome: | The proposed method achieves state-of-the-art performance with five models based on bi-encoder and tri-encoding architectures. |
Advancing Semantic Textual Similarity Modeling: A Regression Framework with Translated ReLU and Smooth K2 Loss (2024.emnlp-main)
Copied to clipboard
| Challenge: | despite its efficiency, Sentence-BERT ignores the progressive nature of semantic relationships, despite a promising approach . contrastive learning methods have improved performance on renowned STS benchmarks, but they fail to leverage fine-grained information. |
| Approach: | They propose a regression framework that categorizes text pairs as either semantically similar or dissimilar . they propose two loss functions: Translated ReLU and Smooth K2 Loss to bridge this gap . |
| Outcome: | The proposed method achieves convincing performance across seven established STS benchmarks. |
SEAVER: Attention Reallocation for Mitigating Distractions in Language Models for Conditional Semantic Textual Similarity Measurement (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Conditional Semantic Textual Similarity (C-STS) introduces specific limiting conditions to the traditional Semantics task. |
| Approach: | They propose a conditional semantic textual similarity (C-STS) task that introduces specific limiting conditions to the traditional Semantic Textual Similarity task. |
| Outcome: | The proposed model outperforms existing models on the C-STS-2023 test set and consistently improves on million-scale fine-tuning baseline models (up to 3 points). |
MUSTS: MUltilingual Semantic Textual Similarity Benchmark (2025.acl-short)
Copied to clipboard
| Challenge: | Existing benchmarks for semantic textual similarity (STS) are limited to high-resource languages and do not include datasets annotated focusing on relatedness instead of similarity. |
| Approach: | They propose to evaluate multilingual semantic textual similarity benchmarks which span 13 languages and annotated datasets to evaluate and compare them. |
| Outcome: | The proposed method is the most comprehensive benchmark of multilingual STS methods. |
Rethinking STS and NLI in Large Language Models (2024.findings-eacl)
Copied to clipboard
| Challenge: | Recent years have seen the rise of large language models (LLMs), where practitioners use task-specific prompts; this was shown to be effective for a variety of tasks. |
| Approach: | They propose to rethink semantic textual similarity (STS) and natural language inference (NLI) models with task-specific prompts and model overconfidence to capture disagreements between human judgements. |
| Outcome: | The proposed models are able to capture human opinions on individual examples without any parameter modifications. |
Going Beyond Sentence Embeddings: A Token-Level Matching Algorithm for Calculating Semantic Textual Similarity (2023.acl-short)
Copied to clipboard
| Challenge: | Semantic Textual Similarity (STS) measures the degree to which the underlying semantics of paired sentences are equivalent. |
| Approach: | They propose a token-level matching inference algorithm which can be applied on top of any language model to improve its performance on STS task. |
| Outcome: | The proposed method improves the performance of almost all language models, with up to 12.7% gain in Spearman’s correlation. |
Sentence Meta-Embeddings for Unsupervised Semantic Textual Similarity (2020.acl-main)
Copied to clipboard
| Challenge: | Existing word embeddings combine complementary strengths of their components to achieve unsupervised semantic similarity (STS). |
| Approach: | They propose to ensemble pre-trained sentence encoders into sentence meta-embeddings to achieve unsupervised Semantic Textual Similarity (STS) they adapt dimensionality reduction, generalized Canonical Correlation Analysis and cross-view auto-encoders to their work. |
| Outcome: | The proposed method achieves 3.7% to 6.4% Pearson’s r over single-source word embeddings on the STS Benchmark and on the StS12-STS16 datasets. |
From Semantics to Style: A Cross-Dataset Comparative Framework for Sentence Similarity Predictions (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing frameworks for analyzing text embedding models are limited. |
| Approach: | They propose a framework that uses lightweight poolers to analyze STS, PI, and Triplet datasets. |
| Outcome: | The proposed framework shows that the model captures semantic differences between sentences and is consistent across datasets. |