Papers by Mehdi Sadi
Analogy-Guided Evolutionary Pretraining of Binary Word Embeddings (2022.aacl-main)
Copied to clipboard
| Challenge: | Existing binary word embeddings are derived from pretrained real-valued embeddables through different simple transformations, which often break the semantic consistency and the “arithmetic” properties of the embedded words. |
| Approach: | They propose a genetic algorithm to learn binary word embeddings from scratch by preserving the semantic relationships between words and the arithmetic properties of the embeddables themselves. |
| Outcome: | Evaluating 16, 32, and 64-bit word embeddings on Mikolov’s word analogy task shows that 95% of the time, the best fit is ranked in the top 5 most similar words in terms of cosine similarity. |