Papers by Tomer Ullman
One fish, two fish, but not the whole sea: Alignment reduces language models’ conceptual diversity (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing studies suggest large language models can capture certain behavioral patterns, but there are ongoing debates as to whether they are valid replacements for human subjects. |
| Approach: | They propose to use large language models as replacements for humans in behavioral research by relating the internal variability of simulated individuals to the population-level variability. |
| Outcome: | The proposed model can capture human-like conceptual diversity, but it is unclear whether post-training alignment affects models’ internal diversity. |
Comparing the Evaluation and Production of Loophole Behavior in Humans and Large Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study shows that loophole-seeking is frequent and intuitive in children . a large number of models capture the pragmatic understanding required for loopholes, says a researcher . |
| Approach: | a study compares large language models to humans to examine loophole behavior . they found that models struggle to recognize humor in creative exploitation of loopholes . |
| Outcome: | a study compares state-of-the-art models to humans to examine loophole behavior in humans . a large language model can generate loopholes, but only two are capable of generating them . |