Papers by Richard Futrell
Language Learning and Processing in People and Machines (N19-5)
Copied to clipboard
| Challenge: | This tutorial introduces different stages of language acquisition and their parallel problems in NLP. |
| Approach: | This tutorial introduces different stages of language acquisition and their parallel problems in NLP. |
| Outcome: | This tutorial introduces different stages of language acquisition and their parallel problems in NLP. |
Structural Supervision Improves Few-Shot Learning and Syntactic Generalization in Neural Language Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have not investigated the relationship between a token's frequency in the training corpus and syntactic properties models learn about it. |
| Approach: | They develop controlled experiments that probe models’ syntactic nominal number and verbal argument structure generalizations for tokens seen as few as two times during training. |
| Outcome: | The proposed models can make syntactic generalizations for tokens seen as few as two times during training and transfer them to transformed contexts. |
Predicting cross-linguistic adjective order with information gain (2021.findings-acl)
Copied to clipboard
| Challenge: | Languages that allow multiple sequential adjective modifiers tend to exhibit strong intralanguage tendencies on the relative order of adjectives. |
| Approach: | They propose a quantitative account of adjective order across typologically-distinct languages based on maximizing information gain. |
| Outcome: | The proposed model addresses the left-right asymmetry of French-type ANA sequences without appeal to other mechanisms. |
An Information-Theoretic Characterization of Morphological Fusion (2021.emnlp-main)
Copied to clipboard
| Challenge: | Linguistic typology generally divides synthetic languages into groups based on their morphological fusion. |
| Approach: | They propose to quantify the degree of fusion of morphological features in a surface form . they recapitulate the usual linguistic classifications for concatenative systems . |
| Outcome: | The proposed measure recapitulates the usual classifications for concatenative systems and provides new measures for nonconcatenating ones. |
The Linearity of the Effect of Surprisal on Reading Times across Languages (2023.findings-emnlp)
Copied to clipboard
| Challenge: | a large amount of insight into human language processing can be gleaned by studying word-by-word processing difficulty. |
| Approach: | They extend the study by examining eyetracking corpora of seven languages . they find evidence for superlinearity in some languages, but highly sensitive to language models . |
| Outcome: | The study extends existing studies on english to Danish, Dutch, English, German, Japanese, Mandarin, and Russian. |
Structural Supervision Improves Learning of Non-Local Grammatical Dependencies (N19-1)
Copied to clipboard
| Challenge: | State-of-the-art LSTM language models learn sequential contingencies with some success . LS models fail to learn other non-local grammatical dependencies, however . |
| Approach: | They compare LSTM language models with RNNGs to examine grammatical dependencies . they find that hierarchical supervision improves learning of non-local dependencies. |
| Outcome: | The proposed model outperforms the existing model on non-local dependencies and learns many of the Island Constraints on the filler-gap dependency. |
Exploring the Sensitivity of LLMs’ Decision-Making Capabilities: Insights from Prompt Variations and Hyperparameters (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Prior studies have compared the decision-making abilities of large language models with those of humans from a psychological perspective. |
| Approach: | They examine LLMs' performance on the Horizon decision-making task studied by Binz and Schulz (2023) they observe that the decision- making abilities fluctuate based on input prompts and temperature settings. |
| Outcome: | The results show that LLMs display a human-like exploration–exploitation tradeoff after simple adjustments to the prompt. |
Deep Subjecthood: Higher-Order Grammatical Features in Multilingual BERT (2021.eacl-main)
Copied to clipboard
| Challenge: | a recent study has shown that multilingual BERT encodes sentences in structurally meaningful ways. |
| Approach: | They analyze how morphosyntactic alignment manifests across embedding spaces of languages . they train classifiers to recover subjecthood of mBERT embedds in transitive sentences . |
| Outcome: | The proposed model encodes a high-order grammatical feature of morphosyntactic alignment across languages . the results show that the classifier distributions reflect the morphological alignment of their training languages based on the results . |
Evaluating a Century of Progress on the Cognitive Science of Adjective Ordering (2023.tacl-1)
Copied to clipboard
| Challenge: | a new study examines the performance of cognitive hypotheses for adjective ordering in 32 languages . linguists and cognitive scientists have proposed an array of hypothese predicting adjective ordering . |
| Approach: | They compare the combined performance of existing adjective ordering proposals across 32 languages . they propose to use a baseline that reflects random chance accuracy and a higher baseline that measures idealized order . |
| Outcome: | The proposed hypotheses are compared with baselines in 32 languages and with random and idealized baselines. |
A Cross-Linguistic Pressure for Uniform Information Density in Word Order (2023.tacl-1)
Copied to clipboard
Thomas Hikaru Clark, Clara Meister, Tiago Pimentel, Michael Hahn, Ryan Cotterell, Richard Futrell, Roger Levy
| Challenge: | a recent study has compared real and counterfactual word orders, but one functional pressure has been overlooked . a study of 10 typologically diverse languages shows that real word orders have greater uniformity than reverse word orders . |
| Approach: | They propose to test whether a pressure for UID may have influenced word order patterns cross-linguistically. |
| Outcome: | The proposed model shows that real orders have greater uniformity than reverse orders among SVO languages. |
When classifying grammatical role, BERT doesn’t care about word order... except when it matters (2022.acl-short)
Copied to clipboard
| Challenge: | Recent work has shown large language models are surprisingly word order invariant . however, word order knowledge is crucial in defining later-layer representations of words . |
| Approach: | They probe grammatical role representations in English BERT and GPT-2 to find word order crucial . they find word orders are crucial in defining later-layer representations of words in non-prototypical positions . |
| Outcome: | The proposed model is based on natural prototypical inputs where word order is crucial for correct classification. |
Simpler neural networks prefer subregular languages (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Inductive biases of neural networks are still poorly understood, says dr. johansen . subregular languages are thought to form a bound on human phonological patterns . |
| Approach: | They apply a relaxation of L0 regularization which induces sparsity to study inductive biases of LSTMs. |
| Outcome: | The proposed method is based on a relaxation of L0 regularization, which induces sparsity, and a subregular language bias in LSTMs is related to the cognitive bias observed in human phonology. |
Neural language models as psycholinguistic subjects: Representations of syntactic state (N19-1)
Copied to clipboard
| Challenge: | a recent study examines the extent to which neural network language models reflect incremental representations of syntactic state . we examine neural network model behavior on sentences chosen to probe specific aspects of the learned representations . |
| Approach: | They employ experimental methodologies developed in psycholinguistics to study syntactic representation in the human mind. |
| Outcome: | The proposed models are trained on large datasets and only sensitive to subtle cues . the results raise questions about the accuracy of the models and their performance . |
Measuring Morphological Fusion Using Partial Information Decomposition (2022.coling-1)
Copied to clipboard
| Challenge: | agglutinative and fusional languages have a systematic relationship between meaning and form, but are less systematic when it comes to morphological relations. |
| Approach: | They propose a mathematically precise way of characterizing morphological systems using partial information decomposition. |
| Outcome: | The proposed framework decomposes mutual information into three components: unique, redundant, and synergistic information. |
The Natural Stories Corpus (L18-1)
Copied to clipboard
Richard Futrell, Edward Gibson, Harry J. Tily, Idan Blank, Anastasia Vishnevetsky, Steven Piantadosi, Evelina Fedorenko
| Challenge: | Existing corpora of naturalistic text do not contain the low-frequency syntactic constructions needed to distinguish between theories. |
| Approach: | They propose to compare models of language processing by comparing their ability to predict behavioral and neural measures of processing difficulty to corpora of naturalistic text. |
| Outcome: | The proposed corpus contains low-frequency syntactic constructions while sounding fluent to native speakers. |
Memory efficiency and resource-rational encoding in sentence processing (2026.acl-long)
Copied to clipboard
| Challenge: | Existing studies have shown that language models need to be constrained in their use of working memory for context, the analogue to human working memory (WM). |
| Approach: | They propose to inject noise into hidden representations of Transformer-based LMs to capture constraint on memory encoding. |
| Outcome: | The proposed model improves alignment with human reading times and makes them more compressed and categorical. |
What determines the order of adjectives in English? Comparing efficiency-based theories using dependency treebanks (2020.acl-main)
Copied to clipboard
| Challenge: | Across languages, there exist strong and stable constraints on the order of adjectives when multiple adjectives modify a noun . adverb order is a crucial testing ground for quantitative theories of syntax . |
| Approach: | They propose four quantitative theories that are motivated by efficiency in human language production and comprehension. |
| Outcome: | The proposed theories predict order of adjectives in hand-parsed and automatically-parsed dependency treebanks. |
Sensitivity as a Complexity Measure for Sequence Classification Tasks (2021.tacl-1)
Copied to clipboard
| Challenge: | Existing complexity metrics provide limited practical insight into complexity differences between tasks. |
| Approach: | They propose a theoretical framework for understanding and predicting the complexity of sequence classification tasks using a new extension of the theory of Boolean function sensitivity. |
| Outcome: | The proposed framework predicts the complexity of sequence classification tasks using a new method . it shows that low-sensitivity functions are easier to learn for LSTMs than lexical classifiers . |