| Challenge: | Recent work shows that text-based language models predict both text- and speech-evoked brain activity. |
| Approach: | They remove low-level stimulus features from language models to assess their impact on alignment with fMRI brain recordings during reading and listening. |
| Outcome: | The proposed model removes low-level features from fMRI brain recordings to assess their impact on alignment with fmr recordings. |
Similar Papers
Language models and brains align due to more than next-word prediction and word-level information (2024.emnlp-main)
Copied to clipboard
| Challenge: | Pretrained language models have been shown to significantly predict brain recordings of people comprehending language. |
| Approach: | They propose to use two perturbations to design contrasts that control for different types of information. |
| Outcome: | The proposed model is largely agnostic about the exact linguistic information contained in the conceptual quantities "word-level information" and "multi-word information". |
Aligning Text/Speech Representations from Multimodal Models with MEG Brain Activity During Listening (2025.emnlp-main)
Copied to clipboard
Padakanti Srijith, Khushbu Pahwa, Radhika Mamidi, Bapi Raju Surampudi, Manish Gupta, Subba Reddy Oota
| Challenge: | Recent studies have found that speech language models fail to capture brain-relevant semantics beyond low-level features. |
| Approach: | They analyze multimodal models to assess their alignment with MEG brain recordings . they find text embeddings from multimodal and unimodal models significantly outperform unilateral models . |
| Outcome: | a new study shows that text-based models outperform unimodal models in alignment with brain recordings during naturalistic story listening. |
Encoding and Decoding Language in the Brain with Language Models (2026.eacl-tutorials)
Copied to clipboard
| Challenge: | This tutorial introduces brain-language model alignment and recent advances in brain-informed fine-tuning and brain-based fine-caching with language models. |
| Approach: | This tutorial introduces brain-language model alignment and recent advances in brain-informed fine-tuning and scaling with language models. |
| Outcome: | This tutorial introduces brain-language model alignment and recent advances in brain-informed fine-tuning and decoding with language models. |
Model-based analysis of brain activity reveals the hierarchy of language in 305 subjects (2021.findings-emnlp)
Copied to clipboard
| Challenge: | a popular approach to decompose the neural bases of language requires large and costly data sets to obtain. |
| Approach: | They propose a model-based approach to decompose the neural bases of language that can be used to correlate brain responses to different stimuli. |
| Outcome: | The proposed model-based approach replicates the seminal study of Lerner et al. (2011), which revealed the hierarchy of language areas by comparing the functional-magnetic resonance imaging (fMRI) of seven subjects listening to 7min of both regular and scrambled narratives. |
Unveiling Multi-level and Multi-modal Semantic Representations in the Human Brain using Large Language Models (2024.emnlp-main)
Copied to clipboard
Yuko Nakagi, Takuya Matsuyama, Naoko Koide-Majima, Hiroto Yamaguchi, Rieko Kubo, Shinji Nishimoto, Yu Takagi
| Challenge: | Recent studies have assessed different levels of semantic content, such as speech, objects, and stories, separately. |
| Approach: | They used functional magnetic resonance imaging to record brain activity while watching 8.3 hours of dramas and movies. |
| Outcome: | The findings show that LLMs predict human brain activity more accurately than traditional language models, particularly for complex background stories. |
Mapping Brains with Language Models: A Survey (2023.findings-acl)
Copied to clipboard
| Challenge: | accumulated evidence for brain and language model activations remains ambiguous, but correlations with model size and quality provide grounds for cautious optimism. |
| Approach: | They examine the evidence accumulated by 30 studies spanning 10 datasets and 8 metrics to determine whether there is any overlap between brain and language model activations. |
| Outcome: | The findings suggest that representations extracted from NLP models can (partially) explain the signal found in neural data. |
Improve Language Model and Brain Alignment via Associative Memory (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing studies have shown that associative memory is essential for language comprehension and comprehension. |
| Approach: | They propose to integrate associative memory into language models to improve alignment . they find alignment is improved in brain regions closely related to associativ memory processing . |
| Outcome: | The proposed model improves in brain regions closely related to associative memory processing. |
From Language to Cognition: How LLMs Outgrow the Human Language Network (2025.emnlp-main)
Copied to clipboard
Badr AlKhamissi, Greta Tuckute, Yingtian Tang, Taha Osama A Binhuraib, Antoine Bosselut, Martin Schrimpf
| Challenge: | Large language models exhibit remarkable similarity to neural activity in the human language network, but their properties remain unclear. |
| Approach: | They benchmark 34 training checkpoints spanning 300B tokens across 8 different model sizes . they find that brain alignment tracks the development of formal linguistic competence more closely than functional linguistic competency. |
| Outcome: | The results show that large language models exhibit similarity to human language networks . they show that the correlation between next-word prediction and brain alignment fades once models surpass human language proficiency. |
From Language to Language-ish: How Brain-Like is an LSTM’s Representation of Nonsensical Language Stimuli? (2020.findings-emnlp)
Copied to clipboard
| Challenge: | LSTMs are often used to measure event related potentials, but are they able to generalize to new data in a human-like way? |
| Approach: | They asked whether an LSTM model represents a language sample with degraded semantic or syntactic information and whether it resembles the brain's reaction to the stimuli. |
| Outcome: | The results suggest that LSTMs and human brain handle nonsensical data similarly. |
Neural Language Taskonomy: Which NLP Tasks are the most Predictive of fMRI Brain Activity? (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing literature has focused on pretrainer-based text-driven brain encoding models . however, few studies have explored the efficacy of task-specific learning of Transformers . |
| Approach: | They propose to use ten popular natural language processing tasks to learn Transformer representations for predicting brain responses. |
| Outcome: | The proposed model predicts brain activity across the whole brain. |