The Shape of Word Embeddings: Quantifying Non-Isometry with Topological Data Analysis (2024.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study shows that word embeddings represent language vocabularies as clouds of d-dimensional points . authors assume that word embedded in different languages are essentially isometric . |
| Approach: | They use persistent homology to measure distances between language pairs from unlabeled embeddings . they construct language phylogenetic trees over 81 Indo-European languages . |
| Outcome: | The proposed tree shows that the embeddings differ from the reference tree. |
Similar Papers
Analyzing the Surprising Variability in Word Embedding Stability Across Languages (2021.emnlp-main)
Copied to clipboard
| Challenge: | Word embeddings are powerful representations that form the foundation of many natural language processing architectures. |
| Approach: | They explore word embedding stability in a wide range of languages to gain insight into their stability. |
| Outcome: | The proposed results provide insights into word embedding stability in English and other languages. |
On the Correlation of Word Embedding Evaluation Metrics (2020.lrec-1)
Copied to clipboard
| Challenge: | Word embeddings are geometrical representations of word paradigmatics and syntagmatics. |
| Approach: | They propose to investigate evaluation metrics on various datasets to find correlations . they propose a fast solution to select the best word embeddings among many others . |
| Outcome: | The proposed method could be used to select the best word embeddings among many others. |
Non-Linearity in Mapping Based Cross-Lingual Word Embeddings (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing work on cross-lingual word embeddings rely on linear mappings, but this assumption is not true for all language pairs. |
| Approach: | They propose a non-linear mapping approach which can find non-linesar relationships between languages by kernel Canonical Correlation Analysis. |
| Outcome: | The proposed approach improves on five language pairs on supervised and self-learning scenarios. |
A Closer Look on Unsupervised Cross-lingual Word Embeddings Mapping (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing methods for word embeddings are limited to a single, unannotated corpus, which means that word representations with similar meaning in distinct languages can be very different. |
| Approach: | They propose an unsupervised method for cross-lingual word embedding mapping that uses stochastic initialization and isometric initialization to verify the method's robustness. |
| Outcome: | The proposed method is robust on different embedding representations and new language pairs, particularly those involving Slavic languages like Polish or Czech. |
Analyzing the Limitations of Cross-lingual Word Embedding Mappings (P19-1)
Copied to clipboard
| Challenge: | Existing methods for cross-lingual word embeddings have limited results . existing methods require little or no cross-linguistic signal to work . |
| Approach: | They compare offline mapping methods to an extension of skip-gram that jointly learns both embedding spaces. |
| Outcome: | The proposed method yields more isomorphic embeddings, is less sensitive to hubness, and achieves stronger results in bilingual lexicon induction. |
What’s in Your Embedding, And How It Predicts Task Performance (C18-1)
Copied to clipboard
| Challenge: | Attempts to find a single technique for general-purpose intrinsic evaluation of word embeddings have so far not been successful. |
| Approach: | They propose a method that quantifies interpretable characteristics of word vector neighborhoods and shows how they correlate with performance on 14 extrinsic and intrinsic task datasets. |
| Outcome: | The proposed approach enables multi-faceted evaluation, parameter search, and generally – a more principled, hypothesis-driven approach to development of distributional semantic representations. |
GRI: Graph-based Relative Isomorphism of Word Embedding Spaces (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing attempts to control relative isomorphism of different spaces fail to consider lexical variations of semantically similar words . Existing methods for building bilingual dictionaries rely on geometric similarity of individual spaces . |
| Approach: | They propose a method that incorporates the impact of lexical variations of semantically similar words into the training objective. |
| Outcome: | The proposed method outperforms existing research by improving the average P@1 by 63.6%. |
Revisiting the Context Window for Cross-lingual Word Embeddings (2020.acl-main)
Copied to clipboard
| Challenge: | Existing approaches to mapping-based cross-lingual word embeddings are based on the assumption that the source and target embeddable spaces are structurally similar. |
| Approach: | They propose to use different context windows to evaluate bilingual word embeddings in various languages, domains, and tasks. |
| Outcome: | The size of both the source and target window improves bilingual lexicon induction, especially on frequent nouns. |
Hierarchical Mapping for Crosslingual Word Embedding Alignment (2020.tacl-1)
Copied to clipboard
| Challenge: | Existing strategies that map word embeddings into a crosslingual space are biased towards the choice of the pivot language. |
| Approach: | They propose to map any two languages into a different middle space by learning mappings across languages in a hierarchical way. |
| Outcome: | The proposed strategy significantly improves vocabulary induction scores in all existing benchmarks and in a new non-English–centered benchmark. |
Do Word Embeddings Capture Spelling Variation? (2020.coling-main)
Copied to clipboard
| Challenge: | Using word embeddings, we analyze spelling variation in word embeds trained on Twitter and Reddit data. |
| Approach: | They propose a new perspective on the analysis of word embeddings by focusing on spelling variation. |
| Outcome: | The proposed analysis shows that word embeddings encode spelling variation patterns of various types to some extent, even when trained using the skipgram model. |