Papers by Hail Hochman
Factual Retrieval in LLMs Is a Redundant, Distributed and Non-Contiguous Process (2026.acl-long)
Copied to clipboard
| Challenge: | Existing work posits that factual knowledge is stored at the last entity token position, but the precise mechanics of how facts are retrieved from model parameters remain unclear. |
| Approach: | They propose an iterative patching protocol to identify a minimal subset of layers necessary for attribute retrieval. |
| Outcome: | The proposed method shows that models possess multiple paths for the same entity and fact, highlighting a high degree of redundancy in attribute computation. |