Papers by Andrea Mogini
Context Copying Modulation: The Role of Entropy Neurons in Managing Parametric and Contextual Knowledge Conflicts (2025.findings-emnlp)
Copied to clipboard
Zineddine Tighidet, Andrea Mogini, Hedi Ben younes, Jiali Mei, Patrick Gallinari, Benjamin Piwowarski
| Challenge: | Recent work has identified in autoregressive transformer models a class of neurons that produce a significant effect on the model output entropy while having an overall moderate impact on the ranking of the predicted tokens. |
| Approach: | They identify a class of neurons that produce significant effects on the model output entropy while having an overall moderate impact on the ranking of the predicted tokens. |
| Outcome: | The entropy neurons suppressed context copying behavior in autoregressive transformer models while having moderate impact on the ranking of predicted tokens. |