Papers by Hyunil Kim
Do All Autoregressive Transformers Remember Facts the Same Way? A Cross-Architecture Analysis of Recall Mechanisms (2025.emnlp-main)
Copied to clipboard
| Challenge: | Prior work has identified MLP modules in early layers as key contributors to factual recall. |
| Approach: | They conduct a comprehensive evaluation of factual recall across several models . they find that Qwen-based models behave differently from previous patterns . |
| Outcome: | The results show that attention modules in the earliest layers contribute more to factual recall than MLP modules. |