Papers by Shinnosuke Isono
An Existence Proof for Neural Language Models That Can Explain Garden-Path Effects via Surprisal (2026.acl-long)
Copied to clipboard
| Challenge: | Surprisal theory claims that difficulty of sentences increases linearly with surprise . a neural LM that can explain garden-path effects cannot be built, says a new study . |
| Approach: | They propose to fine-tune neural LMs to better align surprisal-based reading-time estimates with actual reading times. |
| Outcome: | a new study shows that fine-tuned neural LMs do not overfit on held-out items . the results show that they improve predictive power for human reading times . |
If Attention Serves as a Cognitive Model of Human Memory Retrieval, What is the Plausible Memory Representation? (2025.acl-long)
Copied to clipboard
| Challenge: | Recent work in computational psycholinguistics has revealed intriguing parallels between attention mechanisms and human memory retrieval, focusing primarily on vanilla Transformers that operate on token-level representations. |
| Approach: | They propose that the attention mechanism of Transformer Grammar (TG) can serve as a cognitive model of human memory retrieval using Normalized Attention Entropy (NAE) they propose that TG's attention can implement a human memory-retrieval theory known as cue-based retrieval . |
| Outcome: | The attention mechanism of Transformer Grammar (TG) achieves superior predictive power for self-paced reading times compared to vanilla Transformer’s, with further analyses revealing independent contributions from both models. |
Timesteps of Mamba Align with Human Reading Times (2026.findings-acl)
Copied to clipboard
| Challenge: | In Mamba, the recurrent state transition at each layer conceptually takes some duration of time, the discretization timestep t, determined dynamically in response to the input. |
| Approach: | They propose to align per-word processing time in a popular state-space language model Mamba with human reading time using a naturalistic reading dataset. |
| Outcome: | The proposed model can predict reading times comparable to baselines such as word frequency and GPT-2 surprisal and significant even when they are controlled for. |