Papers by Andreas Schramm
How LLMs Comprehend Temporal Meaning in Narratives: A Case Study in Cognitive Evaluation of LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Large language models exhibit increasingly sophisticated linguistic capabilities, yet the extent to which these models reflect human-like cognition versus advanced pattern recognition remains an open question. |
| Approach: | They conduct a series of targeted experiments to assess whether LLMs construct semantic representations and pragmatic inferences in a human-like manner. |
| Outcome: | The proposed framework can be used to assess the cognitive and linguistic capabilities of large language models (LLMs). |
Strong Memory, Weak Control: An Empirical Study of Executive Functioning in LLMs (2026.eacl-long)
Copied to clipboard
Karin de Langis, Jong Inn Park, Bin Hu, Khanh Chi Le, Andreas Schramm, Michael C. Mensink, Andrew Elfenbein, Dongyeop Kang
| Challenge: | Working memory is a critical component of human intelligence and executive functioning . it is correlated with performance on various cognitive tasks, including fluid intelligence . |
| Approach: | They apply working memory tasks to large language models to estimate working memory capacity . they find that LLMs exceed normative human scores, but not executive functioning benchmarks . |
| Outcome: | The proposed models do not show higher performance on executive functioning tasks or problem solving benchmarks. |
Mary, the Cheeseburger-Eating Vegetarian: Do LLMs Recognize Incoherence in Narratives? (2026.eacl-long)
Copied to clipboard
Karin De Langis, Püren Öncel, Ryan Peters, Andrew Elfenbein, Laura Kristen Allen, Andreas Schramm, Dongyeop Kang
| Challenge: | Contemporary models of (human) reading comprehension characterize comprehension as a dynamic process in which the reader continually builds and updates representations to maintain coherence and integrate new information with prior knowledge. |
| Approach: | They use a paired narrative dataset to examine the extent to which large language models can reliably separate incoherent and coherent stories. |
| Outcome: | The proposed models do not eliminate the deficits in the model internal state and behavior. |