| Challenge: | Using a computational model of word recognition, listeners combine expectations about upcoming content with incremental sensory evidence. |
| Approach: | They fit this model to scalp EEG signals recorded as subjects passively listened to a fictional story and found that words require more than 150 ms of input to be recognized. |
| Outcome: | The proposed model formalizes this perceptual process in Bayesian decision theory and reveals distinct neural processing of words depending on whether or not they can be quickly recognized. |
Similar Papers
Word Acquisition in Neural Language Models (2022.tacl-1)
Copied to clipboard
| Challenge: | Language models acquire individual words during training, based on unigram token frequencies, before transitioning loosely to bigram probabilities, eventually converging on more nuanced predictions. |
| Approach: | They examine how neural language models acquire individual words during training, extracting learning curves and ages of acquisition for over 600 words on the MacArthur-Bates Communicative Development Inventory. |
| Outcome: | The models follow consistent patterns during training for both unidirectional and bidirectional models, and for both LSTM and Transformer architectures. |
A Neural Model of Adaptation in Reading (D18-1)
Copied to clipboard
| Challenge: | Several studies suggest that readers do adapt their lexical and syntactic predictions to the current context. |
| Approach: | They propose to add a simple adaptation mechanism to a neural language model to improve predictions of reading times. |
| Outcome: | The proposed model improves predictions of human reading times compared to a non-adaptive model. |
Contextualized Word Representations for Reading Comprehension (N18-2)
Copied to clipboard
| Challenge: | Reading comprehension (RC) is a high-level task in natural language understanding that requires reading a document and answering questions about its content. |
| Approach: | They propose to provide a standard neural network for reading a document and answering a question about its content. |
| Outcome: | The proposed model improves on the competitive SQuAD dataset by providing rich contextualized word representations and allowing it to choose between context-dependent and context-independent representations. |
Familiar words but strange voices: Modelling the influence of speech variability on word recognition (2021.eacl-srw)
Copied to clipboard
| Challenge: | Despite the lack of acoustic-phonetic invariance in speech, listeners can reliably recognize spoken words despite the lack aural-phonemic invariancy. |
| Approach: | They propose a deep neural model which is trained to retrieve the meaning of a word given its spoken form, a task which resembles that faced by a human listener. |
| Outcome: | The proposed model is more sensitive to dialectical variation than gender variation and more related to related languages. |
Human Inspired Progressive Alignment and Comparative Learning for Grounded Word Acquisition (2023.acl-long)
Copied to clipboard
| Challenge: | a recent study shows that word acquisition is an efficient, supervised, and continual process. |
| Approach: | They develop a computational process for word acquisition through comparative learning . they frame the acquisition of words as representation-symbol mapping . |
| Outcome: | The proposed method can be used to learn the meaning of a word efficiently and efficiently. |
Coreference-aware Surprisal Predicts Brain Response (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies have shown that coreference resolution is a key component of language processing and has been used to manipulate variables of interest. |
| Approach: | They propose to enable the parser to process subword information that might better approximate human morphological knowledge and extend evaluation of coreference effects from self-paced reading to human brain imaging data. |
| Outcome: | The proposed model enables the parser to process subword information that might better approximate human morphological knowledge and extends evaluation of coreference effects from self-paced reading to human brain imaging data. |
Probabilistic Transformer: A Probabilistic Dependency Model for Contextual Word Representation (2023.findings-acl)
Copied to clipboard
| Challenge: | Syntactic structures were deemed essential in natural language processing . but since the deep learning revolution, NLP has been dominated by neural models that do not consider syntactical structures in their design. |
| Approach: | They propose a model that models latent representations of words in a sentence . they use a conditional random field to model latent and dependency arcs . |
| Outcome: | The proposed model performs competitively to transformers on small to medium sized datasets. |
Scaling in Cognitive Modelling: a Multilingual Approach to Human Reading Times (2023.acl-short)
Copied to clipboard
| Challenge: | Neural language models provide conditional probability distributions over the lexicon that are predictive of human processing times. |
| Approach: | They propose to use a transformer-based model to generate probabilistic estimates that are less predictive of early eye-tracking measurements reflecting lexical access and early semantic integration. |
| Outcome: | The proposed models show that larger models capture late eye-tracking measurements that reflect the full integration of a word into the current language context. |
Language models and brains align due to more than next-word prediction and word-level information (2024.emnlp-main)
Copied to clipboard
| Challenge: | Pretrained language models have been shown to significantly predict brain recordings of people comprehending language. |
| Approach: | They propose to use two perturbations to design contrasts that control for different types of information. |
| Outcome: | The proposed model is largely agnostic about the exact linguistic information contained in the conceptual quantities "word-level information" and "multi-word information". |
Function Words as Statistical Cues for Language Learning (2026.acl-long)
Copied to clipboard
| Challenge: | Existing studies have argued that function words aid learning abstract grammatical knowledge from linear input. |
| Approach: | They examine the statistical distribution of function words and their properties . they show that function words are reliable, diverse, and informative . |
| Outcome: | The results show that function words preserve high frequency, reliable syntactic association, phrase-boundary alignment and are informative to structural dependency. |