Are explicit belief representations necessary? A comparison between Large Language Models and Bayesian probabilistic models (2025.naacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have indirect pragmatic capabilities, but their performance on Theory of Mind tasks is mixed. |
| Approach: | They propose to use Bayesian probabilistic model to make inferences about others' beliefs to predict human belief inference. |
| Outcome: | The proposed model outperforms the Rational Speech Act (RSA) framework in predicting human belief inferences, even though it does not explicitly encode belief representations. |
Similar Papers
Unveiling the Limits of Large Language Models in Inferring Pragmatic Meaning from Non-Verbal Responses (2026.acl-long)
Copied to clipboard
| Challenge: | Existing studies have focused mainly on LLMs' comprehension of verbal behavior, with non-verbal behavior considered only in conjunction with verbal responses. |
| Approach: | They present the first systematic evaluation of LLMs’ ability to infer pragmatic meaning in dialogue consisting solely of non-verbal responses. |
| Outcome: | The proposed model fails to capture non-verbal intent and has accuracy dropping by 60% compared to verbal ones. |
Large Language Models: The Need for Nuance in Current Debates and a Pragmatic Perspective on Understanding (2023.emnlp-main)
Copied to clipboard
| Challenge: | Current Large Language Models (LLMs) are unparalleled in their ability to generate grammatically correct, fluent text. |
| Approach: | They argue that LLMs only parrot statistical patterns in training data and that language learning in LLM cannot inform human language learning. |
| Outcome: | The proposed model can generate grammatically correct, fluent text without requiring human intervention. |
Do LLMs Play Dice? Exploring Probability Distribution Sampling in Large Language Models for Behavioral Simulation (2025.coling-main)
Copied to clipboard
| Challenge: | LLMs are used to emulate sequential decision-making processes of humans . however, their ability to perform probabilistic sampling is limited . |
| Approach: | They propose to use large language models (LLMs) as agents to emulate the sequential decision-making processes of humans represented as Markov decision-makers (MDPs). |
| Outcome: | The proposed models can understand probabilities, but struggle with sampling precision . integrating coding tools can improve sampling precision, but this level of sampling precision still makes it difficult to simulate human behavior as agents. |
Modeling Human Subjectivity in LLMs Using Explicit and Implicit Human Factors in Personas (2024.findings-emnlp)
Copied to clipboard
Salvatore Giorgi, Tingting Liu, Ankit Aich, Kelsey Isman, Garrick Sherman, Zachary Fried, João Sedoc, Lyle Ungar, Brenda Curtis
| Challenge: | Large language models (LLMs) are increasingly being used in human-centered social scientific tasks, such as data annotation, synthetic data creation, and engaging in dialog. |
| Approach: | They propose to prompt LLMs with human-like personas and ask them to answer as if they were a specific human, either explicitly, with exact demographics, political beliefs, and lived experiences, or implicitly via names prevalent in specific populations. |
| Outcome: | The proposed model is based on explicit, explicit, and implicit personas, and fails to show implicit biases. |
Large Language Models Are Partially Primed in Pronoun Interpretation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing studies suggest large language models acquire rich linguistic representations, but little is known about whether they adapt to linguistic biases in a human-like way. |
| Approach: | They examine whether large language models display human-like referential biases using stimuli and procedures from real psycholinguistic experiments. |
| Outcome: | The proposed models display human-like referential biases when exposed to referential patterns in the local context. |
Language Statistics and False Belief Reasoning: Evidence from 41 Open-Weight LMs (2026.acl-long)
Copied to clipboard
| Challenge: | a recent study on mental state reasoning in language models relies on a relatively small sample of closed-source LMs. |
| Approach: | They replicate and extend published work on false belief task by assessing LM mental state reasoning behavior across 41 open-weight models. |
| Outcome: | The results show that large LMs show higher sensitivity and predictive power . they also show that humans and LM models show a bias towards attributing false beliefs . |
Perceptions to Beliefs: Exploring Precursory Inferences for Theory of Mind in Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | While theory of mind (ToM) is naturally developed for humans in childhood, large language models (LLMs) exhibit inconsistency in ToM tasks, despite early reports of successful cases. |
| Approach: | They propose to evaluate human ToM precursors-perception inference and perception-to-belief inference-in large language models (LLMs) by annotating characters’ perceptions on ToMi and FANToM. |
| Outcome: | The proposed method significantly improves LLMs’ performance in false belief scenarios. |
It’s Not What You Say, It’s How You Say It: Evaluating LLM Responses to Expressions of Belief (2026.acl-long)
Copied to clipboard
| Challenge: | a typology is grounded in four linguistically motivated dimensions: form, evidentiality, epistemic stance, and tone. |
| Approach: | They propose a typology to evaluate how different EoBs affect whether models follow context versus prior knowledge. |
| Outcome: | The proposed model systematically evaluates 16 LLMs that differ in architecture, scale, and training stages . human listeners subconsciously interpret the belief based on how it is expressed, i.e., its explicitness, tone, or contextual cues. |
How Language Models Conflate Logical Validity with Plausibility: A Representational Analysis of Content Effects (2026.findings-acl)
Copied to clipboard
| Challenge: | a number of theories have been proposed to account for content effects in large language models, including the dual-process theory of reasoning, but the mechanisms behind content effects remain unclear. |
| Approach: | They propose to encode validity and plausibility concepts in LLMs by aligning them in representational geometry. |
| Outcome: | The proposed model conflates validity and plausibility, and vice versa. |
Conservative Bias in Large Language Models: Measuring Relation Predictions (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) exhibit pronounced conservative bias in relation extraction tasks, often defaulting to no_relation label when an appropriate option is unavailable. |
| Approach: | They systematically evaluate the trade-off between conservative bias and hallucination in relation extraction tasks by using SBERT and LLM prompts to quantify this effect. |
| Outcome: | The proposed model defaults to no_relation label twice as often as hallucination, resulting in significant information loss when reasoning is not explicitly included in the output. |