Papers by Robert Hawkins
Abstract Visual Reasoning with Tangram Shapes (2022.emnlp-main)
Copied to clipboard
| Challenge: | We use tangrams as stimuli in cognitive science to study abstract visual reasoning . pre-trained weights demonstrate limited abstract reasoning, we observe . |
| Approach: | They propose a resource for studying abstract visual reasoning in humans and machines . they use tangram puzzles as stimuli to create an annotated dataset with >1k distinct stimuli . |
| Outcome: | The proposed resource is visually and linguistically richer than previous resources . pre-trained weights demonstrate limited abstract reasoning, the authors note . |
Open-domain clarification question generation without question examples (2021.emnlp-main)
Copied to clipboard
| Challenge: | Currently, natural language inputs are unclear or ambiguous, causing uncertainty in dialogues. |
| Approach: | They propose a framework for building a visually grounded question-asking model capable of producing polar (yes-no) clarification questions to resolve misunderstandings in dialogue. |
| Outcome: | The proposed model can produce polar (yes-no) clarification questions to resolve misunderstandings in a goal-oriented 20 questions game with synthetic and human answerers. |
Mixed-effects transformers for hierarchical adaptation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Language differs dramatically from context to context, but prompting can be ineffective when contexts are sparse, out-of-sample, or extra-textual. |
| Approach: | They propose a mixed-effects transformer approach for learning hierarchically-structured prefixes to account for structured variation in language use. |
| Outcome: | The proposed approach can be extended to transformer-based architectures while generalizing well to unseen contexts. |
Probing BERT’s priors with serial reproduction chains (2022.findings-acl)
Copied to clipboard
| Challenge: | Large neural language models have induced surprisingly human-like linguistic knowledge, from syntactic structure and subtle lexical biases to more insidious social biase and stereotypes. |
| Approach: | They propose to use serial reproduction chains to generate representative samples from popular masked language models like BERT to test their hypothesis. |
| Outcome: | The proposed method is based on theories of iterated learning in cognitive science and can be used to probe masked language models. |
Investigating representations of verb bias in neural language models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Languages typically provide more than one grammatical construction to express certain types of messages. |
| Approach: | They propose a large benchmark dataset containing 50K human judgments for 5K distinct sentence pairs in the English dative alternation. |
| Outcome: | The proposed model outperforms recurrent architectures even under comparable parameter and training settings. |
Causal interventions expose implicit situation models for commonsense language understanding (2023.findings-acl)
Copied to clipboard
| Challenge: | Classical psycholinguistic accounts have suggested that world knowledge enters into language understanding through structured schemas called situation models. |
| Approach: | They apply causal intervention techniques to transformer models to analyze performance on the Winograd Schema Challenge . |
| Outcome: | The proposed model performs well on the Winograd Schema Challenge . |
Simulating Opinion Dynamics with Networks of LLM-based Agents (2024.findings-naacl)
Copied to clipboard
Yun-Shiuan Chuang, Agam Goyal, Nikunj Harlalka, Siddharth Suresh, Robert Hawkins, Sijia Yang, Dhavan Shah, Junjie Hu, Timothy Rogers
| Challenge: | Existing approaches to simulating opinion dynamics often over-simplify human behavior . authors propose refining LLMs with real-world discourse to better simulate evolution of beliefs . |
| Approach: | They propose to use large language models to simulate opinion dynamics in groups of simulated agents . they found that LLM agents produce more accurate information than ABMs . |
| Outcome: | The proposed model can be used to better simulate opinion dynamics in real-world discourses. |