Papers with SBERT
Representing and Clustering Errors in Offensive Language Detection (2025.naacl-srw)
Copied to clipboard
| Challenge: | Sentence-BERT embeddings of Large Language Model (LLM)-generated linguistic features give the most interpretable clustering for Arabic errors. |
| Approach: | They evaluate the K-Means clustering of four text representations for the task of offensive language detection in English and Levantine Arabic. |
| Outcome: | The proposed clustering of four text representations for offensive language detection in English and Levantine Arabic gives the most human-interpretable clustering for English errors and the grouping is mainly based on the targeted group in the text. |
An Unsupervised Sentence Embedding Method by Mutual Information Maximization (2020.emnlp-main)
Copied to clipboard
| Challenge: | Sentence BERT is inefficient for sentence-pair tasks as it needs to evaluate combinatorially many sentence pairs which is very time-consuming. |
| Approach: | They propose a lightweight extension on top of BERT and a self-supervised learning objective to derive meaningful sentence embeddings in an unsupervised manner. |
| Outcome: | The proposed method outperforms baselines on common semantic textual similarity tasks and downstream supervised tasks and achieves performance competitive with supervised methods on various tasks. |
XL-LEXEME: WiC Pretrained Model for Cross-Lingual LEXical sEMantic changE (2023.acl-short)
Copied to clipboard
| Challenge: | Existing approaches to the Word in Context task use cross-encoders, which prevent the possibility of deriving comparable word embeddings. |
| Approach: | They propose a Lexical Semantic Change Detection model that extends SBERT, highlighting the target word in the sentence. |
| Outcome: | The proposed model outperforms the state-of-the-art on the multilingual benchmarks for SemEval-2020 Task 1 - Lexical Semantic Change (LSC) Detection and the RuShiftEval shared task. |
Improving Neural Models for Radiology Report Retrieval with Lexicon-based Automated Annotation (2022.naacl-main)
Copied to clipboard
| Challenge: | Conventional exact or approximate termbased retrieval methods lack the ability of semantic understanding of the clinical as well as language context. |
| Approach: | They combine clinical finding detection with supervised query match learning to train a model . findings are used as queries to train the Sentence-BERT model using triplet loss . |
| Outcome: | The proposed method outperforms existing methods on multiple retrieval benchmarks. |
Locality Preserving Sentence Encoding (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing word embeddings can be used to learn sentence embedds on the sentence level. |
| Approach: | They propose a sentence embedding method that uses the inner product to compute semantic similarity between sentences. |
| Outcome: | The proposed method encodes sentences better in the sense of semantic structures. |
Predicate-Argument Based Bi-Encoder for Paraphrase Identification (2022.acl-long)
Copied to clipboard
| Challenge: | Cross-encoders have been widely applied to sentence pair tasks . bi-encodings have significantly lower computation complexity and are better suited to symmetric tasks. |
| Approach: | They propose a bi-encoder approach to the paraphrase identification task and incorporate predicate-argument information into SBERT. |
| Outcome: | The proposed model outperforms SBERT/SRoBERTa on six paraphrase identification datasets with a minimal increase in parameters. |
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks (D19-1)
Copied to clipboard
| Challenge: | Existing methods for finding similar sentences require multiple inferences . a modern GPU requires 65 hours to find the most similar pair in 10,000 sentences . |
| Approach: | They propose a modification of the pretrained BERT network that uses siamese and triplet networks to derive semantically meaningful sentence embeddings. |
| Outcome: | The proposed method outperforms existing methods on sentence-pair regression tasks. |
Automating Idea Unit Segmentation and Alignment for Assessing Reading Comprehension via Summary Protocol Analysis (2022.lrec-1)
Copied to clipboard
| Challenge: | In second language learning, summaries are among the most popular type of student assignments. |
| Approach: | They propose to revise the annotation guidelines to allow machine implementation of the new annotation guidelines. |
| Outcome: | The proposed algorithm achieves 0.789 precision and 0.844 recall over the L2WS 2021 corpus. |
Boosting Code Summarization by Embedding Code Structures (2022.coling-1)
Copied to clipboard
| Challenge: | Recent work on code summarization relies on structural information from the abstract syntax tree (AST) of source codes. |
| Approach: | They propose a program dependency graph (PDG) that represents the structure of a code more effectively. |
| Outcome: | The proposed model improves the performance of an out-of-domain benchmark dataset and the measure SBERT score. |
Less Mature is More Adaptable for Sentence-level Language Modeling (2025.acl-long)
Copied to clipboard
| Challenge: | Existing studies fine-tune encoders or contrastive learning approaches to learn sentence representations. |
| Approach: | They propose to use sentence-level models to study how sentence representations influence downstream task performance. |
| Outcome: | The proposed models outperform token-level models in terms of time and data efficiency. |
Evaluating the Potential of Language-family-specific Generative Models for Low-resource Data Augmentation: A Faroese Case Study (2024.lrec-main)
Copied to clipboard
| Challenge: | generative language models have shown promising results for translation in zero, one, and fewshot learning settings, among other types of tasks. |
| Approach: | They propose to prompt a generative language model for the Nordic languages for Faroese to English translation in a zero, one, and few-shot setting and challenge its Farose language understanding capabilities on a small dataset. |
| Outcome: | The proposed model can translate Faroese to English in a zero, one, and few-shot setting and then use it to create an annotated Farose semantic textual similarity (STS) dataset. |
Don’t Take This Out of Context!: On the Need for Contextual Models and Evaluations for Stylistic Rewriting (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing stylistic text rewriting methods ignore the context of the text, causing generic, incoherent, and generic outputs. |
| Approach: | They propose a contextual evaluation metric that integrates preceding context into stylistic text rewriting. |
| Outcome: | The proposed metric integrates the preceding textual context into rewriting and evaluation stages . human preferences are better reflected by the proposed criterio and other metrics . |
TRoTR: A Framework for Evaluating the Re-contextualization of Text Reuse (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for detecting text reuse focus on recontextualization . current approaches focus on text reuse across a diachronic corpus . |
| Approach: | They propose a framework that relies on topic relatedness for evaluating the diachronic change of context in which text is reused. |
| Outcome: | The proposed framework evaluates biblical text reuse human-annotated with topic relatedness . it exhibits greater sensitivity to textual similarity than topic relatedity, the authors show . |
Mapping Work Task Descriptions from German Job Ads on the O*NET Work Activities Ontology (2024.lrec-main)
Copied to clipboard
| Challenge: | a new method for mapping job tasks to labor market ontologies is proposed . a top configuration of the method achieved a notable performance improvement . |
| Approach: | They use ontological data with Multiple Negatives Ranking loss to extract job tasks from job postings . they integrate labeled job advertisement data into training to improve their mapping . |
| Outcome: | The proposed method improves on the German job ads and their ontology . it can be used to map job tasks to established labor market ontologies or taxonomies . |
Conservative Bias in Large Language Models: Measuring Relation Predictions (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) exhibit pronounced conservative bias in relation extraction tasks, often defaulting to no_relation label when an appropriate option is unavailable. |
| Approach: | They systematically evaluate the trade-off between conservative bias and hallucination in relation extraction tasks by using SBERT and LLM prompts to quantify this effect. |
| Outcome: | The proposed model defaults to no_relation label twice as often as hallucination, resulting in significant information loss when reasoning is not explicitly included in the output. |
WhiSPA: Semantically and Psychologically Aligned Whisper with Self-Supervised Contrastive and Student-Teacher Learning (2025.acl-long)
Copied to clipboard
Rajath Rao, Adithya V Ganesan, Oscar Kjell, Jonah Luby, Akshay Raghavan, Scott M. Feltman, Whitney Ringwald, Ryan L. Boyd, Benjamin J. Luft, Camilo J. Ruggero, Neville Ryant, Roman Kotov, H. Schwartz
| Challenge: | Current speech encoding pipelines rely on an additional text-based LM to get robust representations of human communication, even though speech-to-text models often have a LM within. |
| Approach: | They propose to align Whisper's latent space with semantic representations from a text autoencoder and lexically derived embeddings of basic psychological dimensions: emotion and personality. |
| Outcome: | The proposed approach surpasses current speech encoders over self-supervised affective tasks and downstream psychological tasks, achieving an error reduction of 73.4% and 83.8%, respectively. |
SPICED: News Similarity Detection Dataset with Multiple Topics and Complexity Levels (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing semantic textual similarity (STS) datasets are not suitable for news similarity detection due to their specificity to a single topic. |
| Approach: | They propose to segment news similarity datasets into topics to improve model training . they propose four different levels of complexity specifically designed for news similarities detection task . |
| Outcome: | The proposed dataset includes seven topics: Crime & Law, Culture & Entertainment, Disasters & Accidents, Economy & Business, Politics / Conflicts, Science & Technology, Sports. |