Papers by Ryo Yoshida
Language Acquisition Device in Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are less data-efficient than humans, and pre-pretraining on synthetic languages has been proposed to close this gap. |
| Approach: | They propose to pre-pretrain on MP-STRUCT, a formal language whose strings encode hierarchical composition, feature-based dependencies, and long-distance displacement via MERGE, AGREE, and MOVE. |
| Outcome: | The proposed model outperforms k-Shuffle Dyck despite not being definable in C-RASP despite being hierarchically expressive and circuit-theoretically learnable . |
Modeling Human Sentence Processing with Left-Corner Recurrent Neural Network Grammars (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing literature is agnostic about a parsing strategy of hierarchical models . a recent study showed that hierarchically model hierarchic structures capture grammatical dependencies much better than RNNs in targeted syntactic evaluations. |
| Approach: | They evaluated three LMs with head-final left-branching structures and Recurrent Neural Network Grammars with top-down and left-corner parsing strategies as hierarchical models. |
| Outcome: | The proposed model outperforms top-down and left-corner models against human reading times in Japanese. |
Developmentally-plausible Working Memory Shapes a Critical Period for Language Acquisition (2025.acl-long)
Copied to clipboard
| Challenge: | Large language models possess general linguistic abilities comparable to humans but their efficiency in language acquisition remains far inferior. |
| Approach: | They propose a method that initially constrains working memory during the early stages of training and gradually relaxes this constraint as learning progresses. |
| Outcome: | The proposed method outperforms conventional methods without memory constraints or with static memory constraints. |
Targeted Syntactic Evaluation on the Chomsky Hierarchy (2024.lrec-main)
Copied to clipboard
| Challenge: | a novel evaluation paradigm for targeted syntactic evaluations is proposed . we create formal languages that abstract four syntaktic phenomena in natural languages . |
| Approach: | They propose a new evaluation paradigm for Targeted Syntactic Evaluations . they create formal languages that abstract syntactical phenomena in natural languages . |
| Outcome: | The proposed evaluation paradigm evaluates language models on language modeling tasks . it shows that they can capture the structural patterns of the (Adj)n NP type formal language . |
An Existence Proof for Neural Language Models That Can Explain Garden-Path Effects via Surprisal (2026.acl-long)
Copied to clipboard
| Challenge: | Surprisal theory claims that difficulty of sentences increases linearly with surprise . a neural LM that can explain garden-path effects cannot be built, says a new study . |
| Approach: | They propose to fine-tune neural LMs to better align surprisal-based reading-time estimates with actual reading times. |
| Outcome: | a new study shows that fine-tuned neural LMs do not overfit on held-out items . the results show that they improve predictive power for human reading times . |
Tree-Planted Transformers: Unidirectional Transformer Language Models with Implicit Syntactic Supervision (2024.findings-acl)
Copied to clipboard
| Challenge: | Syntactic Language Models (SLMs) have difficulty with inference efficiency due to explicit generation of syntactical structures. |
| Approach: | They propose a method to "plant" trees into attention weights of unidirectional Transformer LMs to implicitly reflect syntactic structures of natural language. |
| Outcome: | The proposed method outperforms SLMs on the SyntaxGym benchmark. |
If Attention Serves as a Cognitive Model of Human Memory Retrieval, What is the Plausible Memory Representation? (2025.acl-long)
Copied to clipboard
| Challenge: | Recent work in computational psycholinguistics has revealed intriguing parallels between attention mechanisms and human memory retrieval, focusing primarily on vanilla Transformers that operate on token-level representations. |
| Approach: | They propose that the attention mechanism of Transformer Grammar (TG) can serve as a cognitive model of human memory retrieval using Normalized Attention Entropy (NAE) they propose that TG's attention can implement a human memory-retrieval theory known as cue-based retrieval . |
| Outcome: | The attention mechanism of Transformer Grammar (TG) achieves superior predictive power for self-paced reading times compared to vanilla Transformer’s, with further analyses revealing independent contributions from both models. |
Lower Perplexity is Not Always Human-Like (2021.acl-long)
Copied to clipboard
| Challenge: | Existing efforts to build human-like computational models have focused on English . a cross-lingual evaluation is needed to build such models, but current research has focused on Japanese . |
| Approach: | They re-examine an established generalization that lower perplexity is not always human-like in Japanese . they propose a cross-lingual evaluation to build human-type computational models . |
| Outcome: | The proposed model lacks universality and lower perplexity is not always human-like . the results suggest a cross-lingual evaluation will be necessary to build human-type models . |
Composition, Attention, or Both? (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work suggests that language models implicitly learn syntactic structures of natural language, even though they do not receive explicit syntatic supervision. |
| Approach: | They propose a novel architecture that recursively compose subtrees with a composition function and selectively attend to previous structural information with sc-attention mechanisms. |
| Outcome: | The proposed architecture can induce human-like syntactic generalization by recursive composition and selective attention to previous structural information. |