Papers with IsoScore
IsoScore: Measuring the Uniformity of Embedding Space Utilization (2022.findings-acl)
Copied to clipboard
| Challenge: | Several studies suggest that contextualized word embedding models do not isotropically project tokens into vector space. |
| Approach: | They propose to use a tool to measure isotropy to quantify the degree to which a point cloud uniformly utilizes the ambient vector space. |
| Outcome: | The proposed tool is the only available tool that accurately measures how uniformly distributed variance is across dimensions in vector space. |
Exploring Geometric Representational Disparities between Multilingual and Bilingual Translation Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing work shows that limited modeling capacity is a major contributor to reduced performance in multilingual models. |
| Approach: | They investigate the isotropy of multilingual model decoder representations using intrinsic dimensionality and IsoScore to measure how they utilize the dimensions in their underlying vector space. |
| Outcome: | The proposed model decoder representations are less isotropic and occupy fewer dimensions than bilingual models. |