Parallax: Visualizing and Understanding the Semantics of Embedding Spaces via Algebraic Formulae (P19-3)
Copied to clipboard
| Challenge: | Embeddings are a fundamental component of many modern machine learning and natural language processing models. |
| Approach: | They propose a tool for visualizing embedding spaces using parametric projections . they demonstrate the power of Parallax and propose % task-oriented approach . |
| Outcome: | The proposed tool is based on two-dimensional projections without interpretable semantics . it enhances interpretability and allows for more fine-grained analysis . |
Similar Papers
Embeddings in Natural Language Processing (2020.coling-tutorials)
Copied to clipboard
| Challenge: | Embeddings have been a key topic of interest in NLP for the past decade . a quick warm-up introduction to NLP and why it is important to have a semantic comprehension of texts . |
| Approach: | This tutorial will provide a high-level synthesis of the main embedding techniques in NLP . it will start with word embedds and then move to other types of embeddable vectors . |
| Outcome: | This tutorial will provide a high-level synthesis of the main embedding techniques in NLP . it will start with word embedds and move to other types of embeddable representations . |
TriEmbed: Bridge the Gap between Text and Token Indices with Embedding Reparameterization (2025.findings-acl)
Copied to clipboard
| Challenge: | a current paradigm of language modeling discards linguistic relations between tokens during tokenization, creating a fundamental gap . empirical results show that TriEmbed provides more linguistically informative token embeddings . |
| Approach: | They propose a reparameterization method that incorporates morphological relationships . they propose to organize the vocabulary into a Trie structure to reparametrize embeddings . |
| Outcome: | Empirical results show that TriEmbed outperforms existing token embeddings while offering more linguistically informative token embeds. |
Discovering Universal Geometry in Embeddings with ICA (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have focused on achieving sparse embeddings or acquiring semantic axes, but this study focuses on the intrinsic independence present within embeddables. |
| Approach: | They propose to use independent component analysis to extract independent semantic components from pre-trained embeddings by leveraging anisotropic information that remains after the whitening process in Principal Component Analysis. |
| Outcome: | The proposed method reveals that embeddings can be expressed as a composition of a few interpretable axes and that these axe axe are consistent across languages, algorithms, and modalities. |
Interpretable Text Embeddings and Text Similarity Explanation: A Survey (2025.emnlp-main)
Copied to clipboard
| Challenge: | Text embeddings are a fundamental component in many NLP tasks, but their interpretation and explanation remain challenging. |
| Approach: | They propose a framework for interpretable text embeddings and text similarity explanation . they characterize the main ideas, approaches, and trade-offs and discuss lessons learned . |
| Outcome: | The proposed methods are compared with existing models and compare them with existing ones. |
CLIX: Cross-Lingual Explanations of Idiomatic Expressions (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing definition generation systems are difficult to use in second language learning due to the presence of unfamiliar words and grammar. |
| Approach: | They propose to use cross-lingual explanations of idiomatic expressions to support vocabulary expansion for language learners. |
| Outcome: | The proposed system is able to explain idiomatic expressions in non-standard language. |
PWESuite: Phonetic Word Embeddings and Tasks They Facilitate (2024.lrec-main)
Copied to clipboard
Vilém Zouhar, Kalvin Chang, Chenxuan Cui, Nate B. Carlson, Nathaniel Romney Robinson, Mrinmaya Sachan, David R. Mortensen
| Challenge: | Existing word embedding methods overlook phonetic information that is crucial for many tasks. |
| Approach: | They propose three methods that use articulatory features to build phonetically informed word embeddings. |
| Outcome: | The proposed methods improve word retrieval and correlation with sound similarity and on rhyme and cognate detection tasks. |
Quantifying Compositionality of Classic and State-of-the-Art Embeddings (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Static word embeddings make strong claims about compositionality, but the SOTA generative models go too far in the other direction. |
| Approach: | a new study evaluates the compositionality of word embeddings by canonical correlation analysis . strong compositional signals are observed in later training stages across data modalities . |
| Outcome: | a new evaluation of compositional models shows that they exploit access meanings when justified . strong compositional signals are observed in later training stages and in deeper layers of the transformer-based model before a decline at the top layer. |
Adjusting Interpretable Dimensions in Embedding Space with Human Judgments (2024.naacl-long)
Copied to clipboard
| Challenge: | Embedding spaces contain interpretable dimensions indicating gender, formality in style, or even object properties. |
| Approach: | They combine seed-based vectors with human ratings of where words fall along a specific dimension to evaluate on predicting object properties and stylistic properties. |
| Outcome: | The proposed model improves on seed-based vectors and human ratings on object properties and stylistic properties. |
The Shape of Word Embeddings: Quantifying Non-Isometry with Topological Data Analysis (2024.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study shows that word embeddings represent language vocabularies as clouds of d-dimensional points . authors assume that word embedded in different languages are essentially isometric . |
| Approach: | They use persistent homology to measure distances between language pairs from unlabeled embeddings . they construct language phylogenetic trees over 81 Indo-European languages . |
| Outcome: | The proposed tree shows that the embeddings differ from the reference tree. |
What’s in Your Embedding, And How It Predicts Task Performance (C18-1)
Copied to clipboard
| Challenge: | Attempts to find a single technique for general-purpose intrinsic evaluation of word embeddings have so far not been successful. |
| Approach: | They propose a method that quantifies interpretable characteristics of word vector neighborhoods and shows how they correlate with performance on 14 extrinsic and intrinsic task datasets. |
| Outcome: | The proposed approach enables multi-faceted evaluation, parameter search, and generally – a more principled, hypothesis-driven approach to development of distributional semantic representations. |