Using LSTMs to Assess the Obligatoriness of Phonological Distinctive Features for Phonotactic Learning (P19-1)
Copied to clipboard
| Challenge: | a recent study has shown that phonetic information is important for phonotactic learning . a feature-naive model outperforms the feature-aware one in terms of probability assigned to a held-out test set of words . |
| Approach: | They compare two recurrent neural network models of phonotactic learning to determine phonetic information in the form of phonological distinctive features. |
| Outcome: | The proposed model outperforms the one with access to distinctive features at the start of learning . the results suggest that distinctive features are not obligatory for learning phonotactic patterns . |
Similar Papers
RNN Simulations of Grammaticality Judgments on Long-distance Dependencies (C18-1)
Copied to clipboard
| Challenge: | LSTM networks can detect linguistic structures which are ungrammatical due to extraction violations, but are sensitive to linguistic processing factors. |
| Approach: | They propose to use LSTM networks to detect ungrammatical sentences by detecting extra arguments and subject-relative clause island violations. |
| Outcome: | The proposed model can correctly classify (un)grammatical sentences, in certain conditions, but is sensitive to linguistic processing factors and unable to induce a more abstract notion of grammaticality. |
Recurrent Neural Network Language Models Always Learn English-Like Relative Clause Attachment (2020.acl-main)
Copied to clipboard
| Challenge: | Language modeling is widely used as pretraining for many tasks involving language processing. |
| Approach: | They extend a standard approach to evaluating language models to cases of valid interpretations . they compare model performance in English and Spanish to show that non-linguistic bias overlaps with syntactic structure in English but not Spanish. |
| Outcome: | The proposed model performance in English and Spanish shows that non-linguistic bias overlaps with syntactic structure in English but not in Spanish. |
The Grammar-Learning Trajectories of Neural Language Models (2022.acl-long)
Copied to clipboard
| Challenge: | In this paper, we show that neural language models with different initialization, architecture, and training data acquire linguistic phenomena in a similar order, despite their different end performance. |
| Approach: | They propose to use mutual inductive bias to study linguistic representations implicit in NLMs. |
| Outcome: | The proposed approach shows that NLMs with different initialization, architecture, and training data acquire linguistic phenomena in a similar order, despite their different end performance. |
Structural Supervision Improves Learning of Non-Local Grammatical Dependencies (N19-1)
Copied to clipboard
| Challenge: | State-of-the-art LSTM language models learn sequential contingencies with some success . LS models fail to learn other non-local grammatical dependencies, however . |
| Approach: | They compare LSTM language models with RNNGs to examine grammatical dependencies . they find that hierarchical supervision improves learning of non-local dependencies. |
| Outcome: | The proposed model outperforms the existing model on non-local dependencies and learns many of the Island Constraints on the filler-gap dependency. |
Priorless Recurrent Networks Learn Curiously (2020.coling-main)
Copied to clipboard
| Challenge: | a recent study shows domain-general recurrent neural networks reproduce human language behaviours . a lack of a unified concept of number agreement between these processes is a limitation of the model . |
| Approach: | They propose to use domain-general recurrent neural networks without explicit linguistic inductive biases to reproduce human language behaviours. |
| Outcome: | The proposed model can learn number agreement within unnatural sentences, the authors show . they show that the model has an effective understanding of singular versus plural for individual sentences . |
Predicting Reference: What do Language Models Learn about Discourse Models? (2020.emnlp-main)
Copied to clipboard
| Challenge: | a growing literature that probes neural language models to assess their latent acquisition of grammatical knowledge has not investigated their acquisition of discourse modeling ability. |
| Approach: | They draw on a psycholinguistic literature that has established how different contexts affect referential biases concerning who is likely to be referred to next. |
| Outcome: | The proposed models do not resemble human language users, the authors show . their models capture the linguistic knowledge required to perform discourse modeling . |
Word Frequency Does Not Predict Grammatical Knowledge in Language Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Neural language models learn the grammatical properties of natural languages to varying degrees of accuracy. |
| Approach: | They focus on subject-verb agreement and reflexive anaphora to investigate whether there are systematic sources of variation in the language models’ accuracy. |
| Outcome: | The proposed model can learn grammatical properties from training data. |
Explaining Character-Aware Neural Networks for Word-Level Prediction: Do They Discover Linguistic Rules? (D18-1)
Copied to clipboard
| Challenge: | Character-level features are used in many natural language processing algorithms but little is known about the character-level patterns they learn. |
| Approach: | They extend contextual decomposition technique to convolutional neural networks and bidirectional long-term memory networks to evaluate and compare these models for morphological tagging on three morphology-dependent languages. |
| Outcome: | The proposed models implicitly discover understandable linguistic rules for morphological tagging on three morphology-dependent languages. |
Emergent morpho-phonological representations in self-supervised speech models (2025.emnlp-main)
Copied to clipboard
| Challenge: | a recent study shows that self-supervised speech models do not represent phonological and morphological phenomena in frequent English noun and verb inflections. |
| Approach: | They study how S3Ms represent phonological and morphological phenomena in English . they propose alternative representational strategies that may support human spoken word recognition . |
| Outcome: | a new study shows that S3M models can represent phonological and morphological phenomena in English . the models can be trained to recognize spoken words in naturalistic, noisy environments . |
Studying the Inductive Biases of RNNs with Synthetic Variations of Natural Languages (N19-1)
Copied to clipboard
| Challenge: | Recent studies have identified both strengths and limitations of recurrent neural networks (RNNs) in applied natural language processing tasks. |
| Approach: | They propose a paradigm that addresses typological differences between languages . they create synthetic versions of English and train them to predict agreement features . |
| Outcome: | The proposed model improves on predicting agreement with subject and object, suggesting that RNNs have a recency bias. |