Learning Semantic Textual Similarity via Topic-informed Discrete Latent Variables (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent discrete latent variable models have received a surge of interest in both NLP and CV . they are comparable to the continuous counterparts in representation learning, but are more interpretable in their predictions. |
| Approach: | They develop a topic-informed discrete latent variable model for semantic textual similarity . they inject the quantized representation into a transformer-based language model . |
| Outcome: | The proposed model outperforms strong baselines in semantic textual similarity tasks. |
Similar Papers
Long Text Generation with Topic-aware Discrete Latent Variable Model (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent work focuses on the modeling of discourse relation, resulting in discrete codes learning shallow semantics. |
| Approach: | They propose a topic-aware latent code-guided text generation model that encourages discrete codes to model information about topics. |
| Outcome: | The proposed model generates more topic-relevant and coherent texts. |
tBERT: Topic Models and BERT Joining Forces for Semantic Similarity Detection (2020.acl-main)
Copied to clipboard
| Challenge: | Recent pretrained contextual representations such as ELMo and BERT have led to impressive performance gains across a variety of NLP tasks, including semantic similarity detection. |
| Approach: | They propose a topic-informed BERT-based architecture for pairwise semantic similarity detection that adds topic information to pretrained contextual representations such as BERT. |
| Outcome: | The proposed model outperforms existing models on a variety of English language datasets and is highly performant. |
CAST: Corpus-Aware Self-similarity Enhanced Topic modelling (2025.naacl-long)
Copied to clipboard
Yanan Ma, Chenghao Xiao, Chenhan Yuan, Sabine N Van Der Veer, Lamiece Hassan, Chenghua Lin, Goran Nenadic
| Challenge: | Existing topic modelling methods encode contextual information of documents while ignoring contextual details of candidate centroid words. Existing methods are limited by the contextualization gap. |
| Approach: | They propose a topic modelling method that builds upon candidate centroid word embeddings contextualized on the dataset and a self-similarity-based method to filter out less meaningful tokens. |
| Outcome: | The proposed method significantly enhances the coherence and diversity of generated topics, and handles noisy data, outperforming strong baselines. |
A Neural Generative Model for Joint Learning Topics and Topic-Specific Word Embeddings (2020.tacl-1)
Copied to clipboard
| Challenge: | Experimental results show that the proposed model outperforms word-level embedding methods in word similarity evaluation and word sense disambiguation. |
| Approach: | They propose a generative model that explores local and global context for joint learning topics and topic-specific word embeddings. |
| Outcome: | The proposed model outperforms word-level embedding methods in word similarity evaluation and word sense disambiguation. |
A Bilingual Generative Transformer for Semantic Sentence Embedding (2020.emnlp-main)
Copied to clipboard
| Challenge: | Semantic sentence embedding models encode natural language sentences into vectors, such that closeness in embeddable space indicates closeness of semantics between the sentences. |
| Approach: | They propose a deep latent variable model that attempts to perform source separation on parallel sentences, isolating what they have in common in a latent semantic vector, and explaining what is left over with language-specific latent vectors. |
| Outcome: | The proposed model outperforms the state-of-the-art on a standard suite of unsupervised semantic similarity evaluations. |
Pcc-tuning: Breaking the Contrastive Learning Ceiling in Semantic Textual Similarity (2024.emnlp-main)
Copied to clipboard
| Challenge: | Semantic Textual Similarity (STS) is a key indicator of the encoding capabilities of embedding models. |
| Approach: | They propose to use Pearson’s correlation coefficient as a loss function to refine model performance beyond contrastive learning to achieve a Spearman’s ceiling. |
| Outcome: | The proposed method surpasses state-of-the-art strategies with minimal amount of fine-grained annotated samples. |
Mitigating Data Sparsity for Short Text Topic Modeling by Topic-Semantic Contrastive Learning (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing topic models do not make full use of word co-occurrence information to model latent topics. |
| Approach: | They propose a novel short text topic modeling framework, Topic-Semantic Contrastive Topic Model, which uses augmented data and the data characteristic to learn the relations among samples. |
| Outcome: | The proposed framework outperforms state-of-the-art baselines regardless of the data augmentation availability, producing high-quality topics and topic distributions. |
Matching Varying-Length Texts via Topic-Informed and Decoupled Sentence Embeddings (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing approaches to matching text with non-comparable lengths are limited due to truncation issues. |
| Approach: | They propose a model that decouples sentences and embeds them into natural sentences for matching texts of significantly different lengths. |
| Outcome: | The proposed model matches texts of significantly different lengths across three well-studied datasets. |
Explicit Bayesian Inference to Uncover the Latent Themes of Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have impressive generative capabilities, yet their inner mechanisms remain largely opaque. |
| Approach: | They propose a variational autoencoder-based neural topic model to interpret LLMs generation process through an explicit Bayesian framework by inferring latent topic variables via variational inference. |
| Outcome: | The proposed model outperforms state-of-the-art topic models on intrinsic measures of coherence and diversity on multiple datasets and shows significant gains on classification and summarization tasks. |
Fine-grained Contrastive Learning for Definition Generation (2022.aacl-main)
Copied to clipboard
| Challenge: | Recent pre-trained transformer-based definition generation models lack effective representation learning to contain full semantic components of the given word, leading to under-specific definitions. |
| Approach: | They propose a novel contrastive learning method that encourages the model to capture more detailed semantic representations from the definition sequence encoding. |
| Outcome: | The proposed method could generate more specific definitions compared with state-of-the-art models. |