From Imitation to Introspection: Probing Self-Consciousness in Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing language models demonstrate impressive abilities in areas like natural language understanding, content creation, and reasoning. |
| Approach: | They propose a definition of self-consciousness for language models and refine ten core concepts by leveraging structural causal games. |
| Outcome: | The proposed definitions are based on structural causal games and ten core concepts. |
Similar Papers
Overestimation of Syntactic Representation in Neural Language Models (2020.acl-main)
Copied to clipboard
| Challenge: | Several testing methodologies have been developed to probe models’ syntactic representations. |
| Approach: | They propose a method to determine syntactic structure by training a model on strings generated according to a template and testing its ability to distinguish between similar ones with different syntax. |
| Outcome: | The proposed method reproduces positive results with two non-syntactic baseline language models: an n-gram model and an LSTM model trained on scrambled inputs. |
Model-based analysis of brain activity reveals the hierarchy of language in 305 subjects (2021.findings-emnlp)
Copied to clipboard
| Challenge: | a popular approach to decompose the neural bases of language requires large and costly data sets to obtain. |
| Approach: | They propose a model-based approach to decompose the neural bases of language that can be used to correlate brain responses to different stimuli. |
| Outcome: | The proposed model-based approach replicates the seminal study of Lerner et al. (2011), which revealed the hierarchy of language areas by comparing the functional-magnetic resonance imaging (fMRI) of seven subjects listening to 7min of both regular and scrambled narratives. |
AwarenessBench: Assessing Cognitive Capabilities of Language Models (2026.acl-long)
Copied to clipboard
Xiaojian Li, Rongwu Xu, Tianyun Zhang, Yue Wang, Shuo Chen, Qiner Lyu, Briana Zhang, Peiran Yang, Kyle Xue Chen, Haoyuan Shi, Yu Wang, Wei Xu
| Challenge: | Language models exhibit increasingly consciousness-like behaviors, requiring a baseline to evaluate their cognitive abilities. |
| Approach: | They propose a benchmark to assess the cognitive abilities of language models (LMs) they compare 18 state-of-the-art LMs to human models in metacognition, self-awareness, social awareness and situational awareness . |
| Outcome: | Evaluating 18 state-of-the-art LMs, they find they consistently surpass baselines . but most models fall short in metacognition and self-awareness, the study finds . |
Brittle Minds, Fixable Activations: Understanding Belief Representations in Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Despite growing interest in Theory of Mind (ToM) tasks for evaluating language models, little is known about how LMs internally represent mental states of self and others. |
| Approach: | They propose to investigate how LMs internally represent mental states of self and others . |
| Outcome: | The proposed model size and finetuning significantly improve LMs’ internal representations of others’ beliefs, which are structured - not mere by-products of spurious correlations - yet brittle to prompt variations. |
Trustworthiness and Self-awareness in Large Language Models: An Exploration through the Think-Solve-Verify Framework (2024.lrec-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are becoming increasingly influential in reasoning tasks, but they lack trustworthiness and introspective self-awareness when subjected to complex reasoning tasks. |
| Approach: | They propose a framework to explore LLMs’ trustworthiness, introspective self-awareness, and collaborative reasoning by using the Think-Solve-Verify framework. |
| Outcome: | The proposed approach improves from 67.3% to 72.8% on the AQuA dataset and demonstrates the model’s ability to explain the given answers. |
Modeling, Evaluating, and Embodying Personality in LLMs: A Survey (2025.findings-emnlp)
Copied to clipboard
Iago Alves Brito, Julia Soares Dollis, Fernanda Bufon Färber, Pedro Schindler Freire Brasil Ribeiro, Rafael Teixeira Sousa, Arlindo Rodrigues Galvão Filho
| Challenge: | This survey provides a comprehensive overview of the LLM-driven personality scenario. |
| Approach: | This survey provides a comprehensive overview of the LLM-driven personality scenario. |
| Outcome: | The proposed taxonomy analyzes the limitations of existing methods and identifies key research gaps. |
Understanding the Inner-workings of Language Models Through Representation Dissimilarity (2023.emnlp-main)
Copied to clipboard
| Challenge: | Dissimilarity measures measure the extent to which two model’s internal representations differ . they can identify and locate generalization properties of models that are invisible via in-distribution test set performance. |
| Approach: | They propose to use representation dissimilarity measures to measure the extent to which two model’s internal representations differ. |
| Outcome: | The proposed dissimilarity measures can identify and locate generalization properties of models that are invisible via in-distribution test set performance and new evaluations of how language model features vary as width and depth are increased. |
Interactive Evolution: A Neural-Symbolic Self-Training Framework For Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods to fine-tune Large Language Models without human annotations are lacking in the field of natural language training. |
| Approach: | They propose an environment-guided neural-symbolic self-training framework to overcome two main challenges: the scarcity of symbolic data and the limited proficiency of LLMs in processing symbolic language. |
| Outcome: | The proposed framework overcomes two main challenges: the scarcity of symbolic data, and the limited proficiency of LLMs in processing symbolic language. |
How Do Language Models Acquire Character-Level Information? (2026.eacl-long)
Copied to clipboard
| Challenge: | Language models (LMs) implicitly encode character-level information, despite not being explicitly provided during training. |
| Approach: | They analyze how language models acquire character-level knowledge by comparing them to standard settings. |
| Outcome: | The results show that LMs do not treat words as opaque tokens, but instead treat them as tokens. |
SELF-PERCEPT: Introspection Improves Large Language Models’ Detection of Multi-Person Mental Manipulation in Conversations (2025.acl-short)
Copied to clipboard
Danush Khanna, Pratinav Seth, Sidhaarth Sredharan Murali, Aditya Kumar Guru, Siddharth Shukla, Tanuj Tyagi, Sandeep Chaurasia, Kripabandhu Ghosh
| Challenge: | Mental manipulation is subtle yet pervasive form of abuse in interpersonal communication, making its detection critical for safeguarding potential victims. |
| Approach: | They propose a dataset of 220 multi-turn, multi-person dialogues balanced between manipulative and non-manipulative interactions drawn from reality shows that mimic real-life scenarios. |
| Outcome: | The proposed framework shows that it can detect multi-person, multi-turn mental manipulation in multi-people conversations. |