Papers by Gregory Kimmel
Stolen Probability: A Structural Weakness of Neural Language Models (2020.acl-main)
Copied to clipboard
| Challenge: | Neural Network Language Models (NNLMs) generate probability distributions by using a softmax function. |
| Approach: | They propose to use a dot-product distance metric to generate probability distributions . they show that the dot product distance meter introduces a limitation bounding expressiveness . |
| Outcome: | The proposed model optimizes well with the inductive bias of NNLMs, but it results in suboptimal ordering of the embedding space. |