Theory of Mind in Freely-Told Children’s Narratives: A Classification Approach (2023.findings-acl)
Copied to clipboard
| Challenge: | Language and Theory of Mind (ToM) competences are often studied with younger children and standardized tests, but as both are social competences, data and methods with higher ecological validity are critical. |
| Approach: | They leveraged a corpus of 442 freely-told stories by Dutch children aged 4-12 to study language and ToM with NLP-tools. |
| Outcome: | The proposed method is robust relative to the complexity of the task for humans and is consistent with previous studies. |
Similar Papers
Views Are My Own, but Also Yours: Benchmarking Theory of Mind Using Common Ground (2024.findings-acl)
Copied to clipboard
Adil Soubki, John Murzaku, Arash Yousefi Jordehi, Peter Zeng, Magdalena Markowska, Seyed Abolghasem Mirroshandel, Owen Rambow
| Challenge: | Existing benchmarks for theory of mind (ToM) use synthetic data, which can misalign with human behavior. |
| Approach: | They propose a question-answer benchmark based on naturally occurring spoken dialogs to evaluate theory of mind capabilities of language models. |
| Outcome: | The proposed dataset shows that LMs struggle to demonstrate theory of mind (ToM) . |
MindGames: Targeting Theory of Mind in Large Language Models with Dynamic Epistemic Modal Logic (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Theory of Mind (ToM) is a critical component of intelligence but its assessment remains the subject of heated debates. |
| Approach: | They propose to use dynamic epistemic logic to isolate a particular component of ToM and generate controlled problems in English natural language. |
| Outcome: | The proposed language model scales from 70M to 6B and 350M to 174B do not consistently yield better results than random chance. |
The Essence of Contextual Understanding in Theory of Mind: A Study on Question Answering with Story Characters (2025.acl-long)
Copied to clipboard
Chulun Zhou, Qiujing Wang, Mo Yu, Xiaoqian Yue, Rui Lu, Jiangnan Li, Yifan Zhou, Shunchi Zhang, Jie Zhou, Wai Lam
| Challenge: | Theory-of-Mind (ToM) is a psychological capability that allows humans to understand and interpret the mental states of others. |
| Approach: | They propose a CharToM-QA benchmark to assess the importance of comprehensive contextual understanding about personal backgrounds in ToM. |
| Outcome: | The proposed model outperforms existing models on 1,035 ToM questions based on classic novels and shows that educated participants perform better when they have read the novels than non-educated participants. |
Theory of Mind in Large Language Models: Assessment and Enhancement (2025.acl-long)
Copied to clipboard
| Challenge: | Theory of Mind (ToM) is a cornerstone of human social intelligence . Large Language Models (LLMs) are increasingly integrated into daily life . |
| Approach: | They analyze evaluation benchmarks and enhancement strategies to evaluate LLMs' ToM capabilities. |
| Outcome: | The proposed and widely used story-based benchmarks and enhancement strategies are used to evaluate LLMs' ToM capabilities. |
XToM: Exploring the Multilingual Theory of Mind for Large Language Models (2026.acl-long)
Copied to clipboard
Chunkit Chan, Yauwai Yim, Hongchuan Zeng, Zhiying Zou, Xinyuan Cheng, Zhifan Sun, Zheye Deng, Kawai Chung, Yuzhuo Ao, Fan Yixiang, Cheng Jiayang, Ercong Nie, Ginny Wong, Helmut Schmid, Hinrich Schuetze, Simon See, Yangqiu Song
| Challenge: | Existing evaluations of ToM in LLMs are limited to English, neglecting the linguistic diversity that shapes human cognition. |
| Approach: | They propose a multilingual benchmark that evaluates ToM across five languages . they find that models excel in multilingual language understanding, but their ToM performance varies across languages. |
| Outcome: | The proposed benchmark evaluates LLMs across five languages and incorporates diverse task scenarios. |
Towards A Holistic Landscape of Situated Theory of Mind in Large Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent inquiries reveal a lack of robust ToM in large language models . current models focus on different aspects of ToM and are prone to shortcuts and data leakage. |
| Approach: | They propose to taxonomize machine ToM into 7 mental state categories and delineate existing benchmarks to identify under-explored aspects of ToM. |
| Outcome: | The proposed model breaks ToM into individual components and treats LLMs as agents physically and socially situated in interactions with humans. |
On Emergent Social World Models — Evidence for Functional Integration of Theory of Mind and Pragmatic Reasoning in Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models (LMs) possess astonishing abilities and prove useful for a plethora of downstream tasks, but controversy persists regarding how to conceptualize their capacities. |
| Approach: | They analyze LMs’ performance across seven subcategories of ToM abilities using a large localizer dataset than used in prior work. |
| Outcome: | The proposed models recruit shared computational mechanisms for general Theory of Mind (ToM) and language-specific pragmatic reasoning on a substantially larger localizer dataset than used in prior work. |
Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMs (2022.emnlp-main)
Copied to clipboard
| Challenge: | We show that one of today’s largest language models lacks this kind of social intelligence out-of-the-box, using two tasks: SocialIQa and ToMi. |
| Approach: | They propose to use social intelligence and Theory of Mind to examine whether modern large-scale language models lack this kind of social intelligence out-of-the-box. |
| Outcome: | The proposed model lacks social intelligence out-of-the-box, and has well-below human accuracies on SocialIQa and ToMi, respectively. |
Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Models (2024.eacl-long)
Copied to clipboard
Natalie Shapira, Mosh Levy, Seyed Hossein Alavi, Xuhui Zhou, Yejin Choi, Yoav Goldberg, Maarten Sap, Vered Shwartz
| Challenge: | Recent work suggests that Large Language Models (LLMs) exhibit Neural Theory-of-Mind (N-ToM) however, prior work reached conflicting conclusions regarding those abilities. |
| Approach: | They examine the extent of Large Language Models’ N-ToM abilities through an extensive evaluation of 6 tasks and find that LLMs struggle with adversarial examples . |
| Outcome: | The proposed metrics show that LLMs exhibit certain N-ToM abilities, but this behavior is far from robust. |
Machine Theory of Mind Needs Machine Validation (2025.findings-acl)
Copied to clipboard
| Challenge: | In recent years there has been an explosion of interest in studying the extent to which language models (LMs) display a theory of mind (ToM) despite the growth of evaluation tools, the extent of evidence for ToM remains unclear. |
| Approach: | They conduct a survey of 16 recent studies aimed at measuring ToM in language models and found that only half do so for patterns only a machine might exploit. |
| Outcome: | The results show that the datasets that show high LM performance on ToM tasks are easier than their peers, likely due to the presence of spurious patterns in the data. |