| Challenge: | Word2Sense embeddings are interpretable, but they are sparse and fast to compute . a unitary rotation can be applied to many of these embeddables retaining their utility for computational tasks while changing the values of individual coordinates. |
| Approach: | They propose an unsupervised method to generate Word2Sense word embeddings that are interpretable. |
| Outcome: | The proposed method compares well with other unsupervised word embeddings on NLP tasks. |
Similar Papers
SensePOLAR: Word sense aware interpretability for pre-trained contextual word embeddings (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing word embedding models lack interpretability for words . |
| Approach: | They propose to add interpretability to word embeddings by using a POLAR framework that enables wordsense aware interpretations for pre-trained contextual word embeds. |
| Outcome: | The proposed framework achieves comparable performance to existing embeddings across GLUE and SQuAD benchmarks. |
Explaining Word Embeddings via Disentangled Representation (2020.aacl-main)
Copied to clipboard
| Challenge: | Disentangled representations are known to represent interpretable factors in separated dimensions. |
| Approach: | They propose to transform dense word vectors into disentangled embeddings with improved interpretability by encoding polysemous semantics separately. |
| Outcome: | The proposed model can be encoded into multiple sub-embeddings or sub-areas and generates more efficient and effective features for natural language processing. |
Embeddings in Natural Language Processing (2020.coling-tutorials)
Copied to clipboard
| Challenge: | Embeddings have been a key topic of interest in NLP for the past decade . a quick warm-up introduction to NLP and why it is important to have a semantic comprehension of texts . |
| Approach: | This tutorial will provide a high-level synthesis of the main embedding techniques in NLP . it will start with word embedds and then move to other types of embeddable vectors . |
| Outcome: | This tutorial will provide a high-level synthesis of the main embedding techniques in NLP . it will start with word embedds and move to other types of embeddable representations . |
Unsupervised Learning of Sentence Embeddings Using Compositional n-Gram Features (N18-1)
Copied to clipboard
| Challenge: | Currently, unsupervised word embeddings are routinely trained on large amounts of raw text data. |
| Approach: | They propose to use unsupervised word embeddings to train distributed representations of sentences. |
| Outcome: | The proposed method outperforms state-of-the-art models on most benchmark tasks and is robust to the produced general-purpose sentence embeddings. |
Which Evaluations Uncover Sense Representations that Actually Make Sense? (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing sense representations fail for human-centric tasks like inspecting a language’s sense inventory. |
| Approach: | They propose a coherence evaluation for sense embeddings and a model optimized for finding interpretable sense representations that are more coherent than existing sense embeds. |
| Outcome: | The proposed model is more coherent than existing sense embeddings and offers comparable word similarities with multisense representations while learning more distinguishable, interpretable senses. |
LDIR: Low-Dimensional Dense and Interpretable Text Embeddings with Relative Representations (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing text embeddings with high dimensions are difficult to trace and interpret. |
| Approach: | They propose low-dimensional and interpretable text embeddings with relative representations that encode semantic meanings in a vector space where similar texts are close together in the representation space. |
| Outcome: | The proposed embeddings outperform existing models on multiple tasks with fewer dimensions and are lowdimensional and dense while maintaining interpretability. |
Interpretable Word Embeddings via Informative Priors (D19-1)
Copied to clipboard
| Challenge: | Existing word embeddings lack interpretability and are unsupervised . this limitation limits their use within computational social science and digital humanities. |
| Approach: | They propose to use informative priors to create interpretable dimensions for probabilistic word embeddings using a priori model. |
| Outcome: | The proposed models capture latent semantic concepts better than or on-par with the current state of the art while maintaining the simplicity and generalizability of priors. |
Interpretable Text Embeddings and Text Similarity Explanation: A Survey (2025.emnlp-main)
Copied to clipboard
| Challenge: | Text embeddings are a fundamental component in many NLP tasks, but their interpretation and explanation remain challenging. |
| Approach: | They propose a framework for interpretable text embeddings and text similarity explanation . they characterize the main ideas, approaches, and trade-offs and discuss lessons learned . |
| Outcome: | The proposed methods are compared with existing models and compare them with existing ones. |
Improved Word Sense Disambiguation Using Pre-Trained Contextualized Word Representations (D19-1)
Copied to clipboard
| Challenge: | Contextualized word representations are effective in downstream tasks such as question answering, named entity recognition, and sentiment analysis. |
| Approach: | They propose to integrate pre-trained contextualized word representations into a neural network that captures the whole sentence and the word representation in the sentence. |
| Outcome: | The proposed approach outperforms the state-of-the-art approach that makes use of non-contextualized word embeddings on multiple benchmark WSD datasets. |
Deep Generative Model for Joint Alignment and Word Representation (N18-1)
Copied to clipboard
| Challenge: | EmbedAlign model embeds words in their complete observed context and learns by marginalisation of latent lexical alignments. |
| Approach: | They exploit translation as a distributional context and embed words as posterior probability densities, rather than point estimates, which allows them to compare words in context using a measure of overlap between distributions. |
| Outcome: | The proposed model performs on a range of lexical semantics tasks and achieves competitive results on benchmarks including natural language inference, paraphrasing, and text similarity. |