| Challenge: | Recent studies show that cognitively motivated "attention" mechanism in neural models is not a good indicator for relative importance. |
| Approach: | They compare the performance of language-specific and multilingual pretrained transformer models to predict reading time measures reflecting natural human sentence processing. |
| Outcome: | The proposed models predict reading time measures on Dutch, English, German, and Russian texts. |
Similar Papers
Scaling in Cognitive Modelling: a Multilingual Approach to Human Reading Times (2023.acl-short)
Copied to clipboard
| Challenge: | Neural language models provide conditional probability distributions over the lexicon that are predictive of human processing times. |
| Approach: | They propose to use a transformer-based model to generate probabilistic estimates that are less predictive of early eye-tracking measurements reflecting lexical access and early semantic integration. |
| Outcome: | The proposed models show that larger models capture late eye-tracking measurements that reflect the full integration of a word into the current language context. |
Measuring the Impact of (Psycho-)Linguistic and Readability Features and Their Spill Over Effects on the Prediction of Eye Movement Patterns (2022.acl-long)
Copied to clipboard
| Challenge: | Existing work to predict gaze patterns during naturalistic reading has not been conducted on general text characteristics. |
| Approach: | They propose to use two eye-tracking corpora of naturalistic reading and two language models to test their performance. |
| Outcome: | The proposed models predict eye-tracking measures during naturalistic reading and language processing. |
Modeling Nonnative Sentence Processing with L2 Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results show that while all of the LMs’ word surprisals improve prediction of L2 reading times, there is no reliable effect of the choice of L1’s L1. |
| Approach: | They pretrain GPT2 on 6 different first languages, followed by English as the second language (L2). |
| Outcome: | The pretraining of L1 improves prediction of L2 reading times, but there is no reliable effect of the pretraining L1 on the model's performance on English speakers. |
From Human Reading to NLM Understanding: Evaluating the Role of Eye-Tracking Data in Encoder-Based Models (2025.acl-long)
Copied to clipboard
| Challenge: | integrating eye-tracking features into Neural Language Models does not degrade downstream task performance, enhances alignment between model attention and human attention patterns, and compresses the embedding space. |
| Approach: | They used eye-gaze data from the Ghent Eye-Tracking Corpus to investigate how integrating knowledge of human reading behavior impacts Neural Language Models. |
| Outcome: | The proposed approach does not degrade downstream task performance, enhances alignment between model attention and human attention patterns, and compresses the embedding space. |
Every word counts: A multilingual analysis of individual human alignment with model attention (2022.aacl-short)
Copied to clipboard
| Challenge: | Using eye-tracking data, fixation durations are often not considered in generalisation studies because of individual differences. |
| Approach: | They analyse eye-tracking data from speakers of 13 different languages reading . they find significant differences between languages but also individual reading behaviour . |
| Outcome: | The proposed model can be used to improve the generalization of ML models and allow for more personalized and fair applications. |
Transformer-Based Language Model Surprisal Predicts Human Reading Times Best with About Two Billion Training Tokens (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies have drawn conflicting conclusions about the relationship between the quality of a language model and the ability of its surprisal estimates to predict human reading times. |
| Approach: | They propose to evaluate surprisal estimates from Transformer-based language model variants that vary systematically in the amount of training data and model capacity on their ability to predict human reading times. |
| Outcome: | The proposed model variants with contemporary model capacities provide the best fit after seeing about two billion training tokens, while smaller models show a ‘tipping point’ at convergence after the decrease in language model perplexity . |
Probing for Reading Times (2026.acl-long)
Copied to clipboard
Eleftheria Tsipidi, Samuel Kiegeland, Francesco Ignazio Re, Tianyang Xu, Mario Giulianelli, Karolina Stanczak, Ryan Cotterell
| Challenge: | a large body of work on probing has demonstrated that language model representations encode a wealth of linguistic information, but it remains unclear whether they also capture cognitive signals about human processing. |
| Approach: | They use regularized linear regression to compare language model representations against scalar predictors. |
| Outcome: | The representations from early layers outperform surprisal in predicting early-pass measures such as first fixation and gaze duration. |
Do Transformer Models Show Similar Attention Patterns to Task-Specific Human Gaze? (2022.acl-long)
Copied to clipboard
| Challenge: | We compare attention functions in pre-trained language models to human eye fixation patterns during task-specific reading tasks. |
| Approach: | They compare attention functions in large-scale pre-trained language models to classical cognitive models of human attention by using a dataset with eye-tracking recordings of native speakers of English. |
| Outcome: | The proposed model is as predictive of human eye fixation patterns as classical cognitive models of human attention. |
Cross-Lingual Transfer of Cognitive Processing Complexity (2023.findings-eacl)
Copied to clipboard
| Challenge: | Recent studies indicate that multilingual language models utilize structural similarities between languages to facilitate cross-lingual transfer. |
| Approach: | They propose a multilingual model that uses structural similarities between languages to facilitate cross-lingual transfer by a meaningful bias towards sentence length and cross-linguistic differences. |
| Outcome: | The proposed model can predict varied patterns for 13 languages, despite being fine-tuned only on English data. |
Assessing the Syntactic Capabilities of Transformer-based Multilingual Language Models (2021.findings-acl)
Copied to clipboard
| Challenge: | Multilingual Transformer-based language models have been shown to be excellent learners in crosslingual transfer tasks. |
| Approach: | They evaluate the syntactic generalization capabilities of BERT and RoBERTa models on English and Spanish tests. |
| Outcome: | The proposed models perform well on English and Spanish tests, and the proposed tests are compared against models on the same language and models on two different languages. |