Papers by John Hale
Do LLMs learn a true syntactic universal? (2024.emnlp-main)
Copied to clipboard
| Challenge: | linguistics literature has debated whether large multilingual language models learn language universals . Typological generalizations are a key battleground in such debates - e.g. van der Hulst, 2023, chapter 7). |
| Approach: | They consider a candidate universal for language universals, the Final-over-Final Condition . they suggest that modern language models may need additional sources of bias to become truly human-like . |
| Outcome: | The proposed model only seems to recognize the Final-over-Final Condition in German, Russian, Hungarian and Serbian . |
Text Genre and Training Data Size in Human-like Parsing (D19-1)
Copied to clipboard
| Challenge: | Using domain-specific training, NLP systems work better, but only when the training examples come from the same textual genre. |
| Approach: | They relate the states of a neural phrase-structure parser to electrophysiological measures from human participants. |
| Outcome: | The proposed model is well-matched to the training data from human participants, but only when the training examples come from the same genre. |
Multipath parsing in the brain (2024.acl-long)
Copied to clipboard
| Challenge: | a major unsolved problem in computational psycholinguistics is determining whether human comprehension considers a single analysis path 1 at a time. |
| Approach: | They compare syntactic surprisal from a state-of-the-art dependency parser with fMRI data . they find evidence for multipath parsing in English and Chinese data based on fm data a major unsolved problem in computational psycholinguistics is determining whether human sentence comprehension considers a single analysis path 1 at a time . |
| Outcome: | The proposed model shows that human parsing is multipath, with a higher r2 increase for multipath surprisal than single-path surpresal. |
An Evaluation of Croatian ASR Models for Čakavian Transcription (2024.lrec-main)
Copied to clipboard
| Challenge: | akavian is an endangered language closely related to Croatian . |
| Approach: | They evaluate four currently available automatic speech recognition systems that are trained on standard Croatian data and assess their performance in the transcription of akavian audio data. |
| Outcome: | The proposed models perform better than the standard conformer model and the best-performing variant of the CTC-based model. |
Finding syntax in human encephalography with beam search (P18-1)
Copied to clipboard
| Challenge: | RNNGs are generative models of (tree , string ) pairs that evaluate derivational choices . a non-syntactic neural language model yields no reliable effects . |
| Approach: | They propose to combine a probabilistic generative grammar with a parsing procedure that uses it to manage syntactic derivations as it advances from one word to the next. |
| Outcome: | The proposed model derives two amplitude effects when used against human encephalography data. |
LSTMs Can Learn Syntax-Sensitive Dependencies Well, But Modeling Structure Makes Them Better (P18-1)
Copied to clipboard
| Challenge: | a recent study found that language models fail to learn long-range syntax sensitive dependencies. |
| Approach: | They propose to use a subject-verb agreement diagnostic to determine whether language models can learn long-range syntax sensitive dependencies. |
| Outcome: | The proposed model outperforms left-corner and bottom-up variants in learning non-local dependencies. |
The Alice Datasets: fMRI & EEG Observations of Natural Language Comprehension (2020.lrec-1)
Copied to clipboard
| Challenge: | "naturalistic" stimuli are now offering a new way to study language comprehension in the brain, in synergy with natural language processing tools. |
| Approach: | They propose to use a set of datasets from a story in English to test new linguistic and computational hypotheses about natural language comprehension in the brain. |
| Outcome: | The Alice Datasets are a set of datasets based on magnetic resonance and electrophysiological data, collected while participants heard a story in English. |