Papers by David Duvenaud
Understanding Undesirable Word Embedding Associations (P19-1)
Copied to clipboard
| Challenge: | Word embeddings are often criticized for capturing undesirable word associations such as gender stereotypes. |
| Approach: | They propose to use subspace projection to debias vectors post hoc using a model that implicitly does matrix factorization to debunk gender bias. |
| Outcome: | The proposed test overestimates gender bias in word embeddings by using subspace projection, a method that is widely used in training. |
Towards Understanding Linear Word Analogies (P19-1)
Copied to clipboard
| Challenge: | Existing theories of word embeddings make strong assumptions about the embeddable space or word distribution. |
| Approach: | They propose a formal explanation of word analogies by adding arithmetic operators to non-linear embedding models such as skip-gram with negative sampling. |
| Outcome: | The proposed model downweights the more frequent word, as weighting schemes do ad hoc. |