Are Embedding Spaces Interpretable? Results of an Intrusion Detection Evaluation on a Large French Corpus (2022.lrec-1)
Copied to clipboard
| Challenge: | Word embedding methods use word co-occurrences to encode, syntactic and semantic information to describe vocabulary in a low-dimensional space. |
| Approach: | They evaluate word embedding interpretability using two methods . they use a word-in-space vector encoder and graph-based method SPINE . |
| Outcome: | The proposed methods show that they can be interpretable on a large French corpus. |
Similar Papers
Adjusting Interpretable Dimensions in Embedding Space with Human Judgments (2024.naacl-long)
Copied to clipboard
| Challenge: | Embedding spaces contain interpretable dimensions indicating gender, formality in style, or even object properties. |
| Approach: | They combine seed-based vectors with human ratings of where words fall along a specific dimension to evaluate on predicting object properties and stylistic properties. |
| Outcome: | The proposed model improves on seed-based vectors and human ratings on object properties and stylistic properties. |
SensePOLAR: Word sense aware interpretability for pre-trained contextual word embeddings (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing word embedding models lack interpretability for words . |
| Approach: | They propose to add interpretability to word embeddings by using a POLAR framework that enables wordsense aware interpretations for pre-trained contextual word embeds. |
| Outcome: | The proposed framework achieves comparable performance to existing embeddings across GLUE and SQuAD benchmarks. |
On the Correlation of Word Embedding Evaluation Metrics (2020.lrec-1)
Copied to clipboard
| Challenge: | Word embeddings are geometrical representations of word paradigmatics and syntagmatics. |
| Approach: | They propose to investigate evaluation metrics on various datasets to find correlations . they propose a fast solution to select the best word embeddings among many others . |
| Outcome: | The proposed method could be used to select the best word embeddings among many others. |
Interpretable Text Embeddings and Text Similarity Explanation: A Survey (2025.emnlp-main)
Copied to clipboard
| Challenge: | Text embeddings are a fundamental component in many NLP tasks, but their interpretation and explanation remain challenging. |
| Approach: | They propose a framework for interpretable text embeddings and text similarity explanation . they characterize the main ideas, approaches, and trade-offs and discuss lessons learned . |
| Outcome: | The proposed methods are compared with existing models and compare them with existing ones. |
Analytical Methods for Interpretable Ultradense Word Embeddings (D19-1)
Copied to clipboard
| Challenge: | Word embeddings lack interpretability, but rotation of word spaces can help . e.g., lexicon induction, gender bias can be removed by removing interpretable dimensions . |
| Approach: | They propose three methods for making word embeddings interpretable by rotation . they use Densifier, linear SVMs and DensRay to compute word spaces in closed form . |
| Outcome: | The proposed method can be computed in closed form and is more robust than Densifier. |
Analyzing the Surprising Variability in Word Embedding Stability Across Languages (2021.emnlp-main)
Copied to clipboard
| Challenge: | Word embeddings are powerful representations that form the foundation of many natural language processing architectures. |
| Approach: | They explore word embedding stability in a wide range of languages to gain insight into their stability. |
| Outcome: | The proposed results provide insights into word embedding stability in English and other languages. |
Are Word Embeddings Really a Bad Fit for the Estimation of Thematic Fit? (2020.lrec-1)
Copied to clipboard
| Challenge: | In recent years, vectors derived from neural network training have replaced count-based distributional semantic models as a de facto standard for word representation in NLP. |
| Approach: | They propose to evaluate count models and word embeddings on thematic fit estimation by taking into account a larger number of parameters and verb roles and introducing dependency-based embedders in the comparison. |
| Outcome: | The proposed model outperforms count models and word embeddings in thematic fit estimation tasks while introducing dependency-based embedders. |
How Well Do Text Embedding Models Understand Syntax? (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing text embedding models have not addressed syntactic understanding challenges, highlighting ineffectiveness and enhancing generalization ability. |
| Approach: | They propose to examine the ability of text embedding models to generalize across syntactic contexts. |
| Outcome: | The proposed models exhibit high similarity socres at this simple task. |
Interpreting Embedding Spaces by Conceptualization (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in large language models have a significant drawback: they are incomprehensible to humans. |
| Approach: | They propose a method for understanding embeddings by transforming a latent embeddable space into a comprehensible conceptual space. |
| Outcome: | The proposed method compares the semantics of the original latent embedding space to the semantic of the vectors. |
Joint Embedding of Words and Labels for Text Classification (P18-1)
Copied to clipboard
Guoyin Wang, Chunyuan Li, Wenlin Wang, Yizhe Zhang, Dinghan Shen, Xinyuan Zhang, Ricardo Henao, Lawrence Carin
| Challenge: | Existing approaches to text classification use word embeddings to capture semantic regularities between words. |
| Approach: | They propose to view text classification as a label-word joint embedding problem . they use a framework that measures compatibility between text sequences and labels . |
| Outcome: | The proposed framework outperforms the state-of-the-art methods on large text datasets. |