Are we there yet? Encoder-decoder neural networks as cognitive models of English past tense inflection (P19-1)
Copied to clipboard
| Challenge: | linguistics and cognitive science have long debated the cognitive mechanisms needed to account for the English past tense. |
| Approach: | They propose to use an encoder-decoder model to account for the english past tense . they also show that ED models demonstrate humanlike performance in a nonce-word task . |
| Outcome: | The proposed model is unstable across simulations and does not fit to human data . other neural models might do better, but there is insufficient evidence to claim them . |
Similar Papers
Inflecting When There’s No Majority: Limitations of Encoder-Decoder Neural Networks as Cognitive Models for German Plurals (2020.acl-main)
Copied to clipboard
| Challenge: | Encoder-decoder models can be used to generalize to inflectional morphology and generalize new words, but they fail on tasks like German number inflection, where infrequent suffixes like /-s/ can still be productively generalized. |
| Approach: | They propose to use a dataset to collect data from German speakers to examine whether ED models can generalize the most frequently produced plural class. |
| Outcome: | The proposed model does not show human-like variability or ‘regular’ extension of other plural markers. |
How do we get there? Evaluating transformer neural networks as cognitive models for English past tense inflection (2022.aacl-main)
Copied to clipboard
| Challenge: | English past tense inflections is a typical quasi-regularity task, but it is criticized that it learns only to generalize the most frequent pattern, but not the regular pattern. |
| Approach: | They train a set of transformer models with different settings to examine their behavior on a typical English quasi-regularity task. |
| Outcome: | The models achieved high accuracy on unseen regular verbs and some accuracy on unseen irregular verbs. |
A Comprehensive Comparison of Neural Networks as Cognitive Models of Inflection (2022.emnlp-main)
Copied to clipboard
| Challenge: | Neural networks are at the center of a debate about human behavior in inflectional morphology. |
| Approach: | They measure correlation between human judgments and neural network probabilities for unknown word inflections. |
| Outcome: | The proposed model for morphological inflections correlates best with human wug ratings, but not with humans. |
Learning Bidirectional Morphological Inflection like Humans (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent research has focused on whether neural models can acquire morphological inflection like humans. |
| Approach: | They propose to use a recurrent neural network with attention and the transformer to train a symbolic model under a human-like learning environment to evaluate their models. |
| Outcome: | The proposed models did not accurately inflect verbs in the same manner as humans in terms of morphological inflection direction. |
Neural language models as psycholinguistic subjects: Representations of syntactic state (N19-1)
Copied to clipboard
| Challenge: | a recent study examines the extent to which neural network language models reflect incremental representations of syntactic state . we examine neural network model behavior on sentences chosen to probe specific aspects of the learned representations . |
| Approach: | They employ experimental methodologies developed in psycholinguistics to study syntactic representation in the human mind. |
| Outcome: | The proposed models are trained on large datasets and only sensitive to subtle cues . the results raise questions about the accuracy of the models and their performance . |
RNN Simulations of Grammaticality Judgments on Long-distance Dependencies (C18-1)
Copied to clipboard
| Challenge: | LSTM networks can detect linguistic structures which are ungrammatical due to extraction violations, but are sensitive to linguistic processing factors. |
| Approach: | They propose to use LSTM networks to detect ungrammatical sentences by detecting extra arguments and subject-relative clause island violations. |
| Outcome: | The proposed model can correctly classify (un)grammatical sentences, in certain conditions, but is sensitive to linguistic processing factors and unable to induce a more abstract notion of grammaticality. |
Are Decoder-Only Language Models Better than Encoder-Only Language Models in Understanding Word Meaning? (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models are highly effective tools for solving different kinds of problems in natural language processing. |
| Approach: | They propose to use large language models to solve a myriad of problems. |
| Outcome: | The proposed model performs worse on word meaning comprehension than an encoder-only model with vastly fewer parameters. |
Recurrent Neural Network Language Models Always Learn English-Like Relative Clause Attachment (2020.acl-main)
Copied to clipboard
| Challenge: | Language modeling is widely used as pretraining for many tasks involving language processing. |
| Approach: | They extend a standard approach to evaluating language models to cases of valid interpretations . they compare model performance in English and Spanish to show that non-linguistic bias overlaps with syntactic structure in English but not Spanish. |
| Outcome: | The proposed model performance in English and Spanish shows that non-linguistic bias overlaps with syntactic structure in English but not in Spanish. |
Chinese Grammatical Correction Using BERT-based Pre-trained Model (2020.aacl-main)
Copied to clipboard
| Challenge: | Recent studies have shown that pre-trained models improve performance on downstream tasks. |
| Approach: | They propose to incorporate a pre-trained model into an encoder-decoder model to improve the performance of Chinese grammatical error correction tasks. |
| Outcome: | The proposed method improves the performance of Chinese grammatical error correction tasks. |
Predicting Reference: What do Language Models Learn about Discourse Models? (2020.emnlp-main)
Copied to clipboard
| Challenge: | a growing literature that probes neural language models to assess their latent acquisition of grammatical knowledge has not investigated their acquisition of discourse modeling ability. |
| Approach: | They draw on a psycholinguistic literature that has established how different contexts affect referential biases concerning who is likely to be referred to next. |
| Outcome: | The proposed models do not resemble human language users, the authors show . their models capture the linguistic knowledge required to perform discourse modeling . |