Papers by Henry Kvinge
Understanding the Inner-workings of Language Models Through Representation Dissimilarity (2023.emnlp-main)
Copied to clipboard
| Challenge: | Dissimilarity measures measure the extent to which two model’s internal representations differ . they can identify and locate generalization properties of models that are invisible via in-distribution test set performance. |
| Approach: | They propose to use representation dissimilarity measures to measure the extent to which two model’s internal representations differ. |
| Outcome: | The proposed dissimilarity measures can identify and locate generalization properties of models that are invisible via in-distribution test set performance and new evaluations of how language model features vary as width and depth are increased. |