Papers by Geoffrey Hinton
Meta-Learning Fast Weight Language Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | Dynamic evaluation of language models (LMs) adapts model parameters at test time using gradient information from previous tokens. |
| Approach: | They propose a neural component that uses gradient updates as linear attention to improve model performance. |
| Outcome: | The proposed model can be applied at training time and learn to make good use of gradient updates. |
Illustrative Language Understanding: Large-Scale Visual Grounding with Image Search (P18-1)
Copied to clipboard
| Challenge: | a large-scale lookup operation to ground language via ‘snapshots’ of our physical world accessed through image search is currently used to learn word representations. |
| Approach: | They propose a large-scale lookup operation to ground language via ‘snapshots’ of our physical world accessed through image search. |
| Outcome: | The proposed model is based on a large-scale lookup operation to ground language using image search. |