Evaluating Subjective Cognitive Appraisals of Emotions from Large Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work on automatic prediction of cognitive appraisals has focused on physiological aspects of emotions. |
| Approach: | They present a dataset that assesses 24 appraisal dimensions across 241 Reddit posts . they find that open-source models fail to automatically assess and explain cognitive appraisals . |
| Outcome: | The proposed dataset assesses 24 appraisal dimensions across 241 reddit posts. |
Similar Papers
The PEACE-Reviews dataset: Modeling Cognitive Appraisals in Emotion Text Analysis (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies have delved into its significance, yet the interplay between various forms of cognitive appraisal and specific emotions, such as joy and anger, remains an area of exploration in consumption contexts. |
| Approach: | They propose to construct a dataset to model the evaluations people make about their situations based on annotated autobiographical accounts of their emotional and appraisal experiences . |
| Outcome: | The proposed model incorporates emotion, cognition, individual traits, and demographic data. |
Modeling Subjectivity in Cognitive Appraisal with Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a new study explores how language models can quantify subjectivity in cognitive appraisal . existing post-hoc calibration methods fail to achieve satisfactory performance . |
| Approach: | They investigate how language models can quantify subjectivity in cognitive appraisal . existing post-hoc calibration methods often fail to achieve satisfactory performance . |
| Outcome: | The proposed model can quantify subjectivity in cognitive appraisal using fine-tuned models and prompt-based large language models. |
Mechanistic Interpretability of Emotion Inference in Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on large language models (LLMs) show promising capabilities in predicting human emotions from text. |
| Approach: | They investigate how autoregressive LLMs infer emotions by focusing on appraisal theory . they show that emotion representations are functionally localized to specific regions in the model . |
| Outcome: | The proposed model is functionally localized to specific regions in the model, and the results align with theoretical and intuitive expectations. |
EmotionQueen: A Benchmark for Evaluating Empathy of Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing evaluations of emotional intelligence in large language models (LLMs) focus on basic sentiment analysis tasks, such as emotion recognition, which is not enough to evaluate LLMs’ overall emotional intelligence. |
| Approach: | They propose a framework for evaluating the emotional intelligence of large language models (LLMs) that includes four distinct tasks: Key Event Recognition, Mixed Event Recognition and Implicit Emotional Recognition. |
| Outcome: | The proposed framework includes four distinct tasks: Key Event Recognition, Mixed Event Recognition and Implicit Emotional Recognition. |
Beyond Context to Cognitive Appraisal: Emotion Reasoning as a Theory of Mind Benchmark for Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have shown that large language models (LLMs) reason about others' emotional states using contextual information, within a Theory-of-Mind framework. |
| Approach: | They propose to use large language models to reason about others’ emotional states using contextual information within a Theory-of-Mind framework. |
| Outcome: | The proposed models can reason about situations and appraisals, but are poor at associating situational outcomes and appraisal with specific emotions. |
Appraisal Theories for Emotion Classification in Text (2020.coling-main)
Copied to clipboard
| Challenge: | Automatic emotion categorization is based on textual units assigned to an emotion from a predefined inventory, for instance following the basic emotion classes proposed by Paul Ekman (1999) or Plutchik (2001). |
| Approach: | They propose to make automatic emotion categorization explicit by following theories of cognitive appraisal of events and show their potential for emotion classification when being encoded in classification models. |
| Outcome: | The proposed models improve the classification of discrete emotion categories by using appraisal dimension assignments in event descriptions. |
EmoBench: Evaluating the Emotional Intelligence of Large Language Models (2024.acl-long)
Copied to clipboard
Sahand Sabour, Siyang Liu, Zheyuan Zhang, June Liu, Jinfeng Zhou, Alvionna Sunaryo, Tatia Lee, Rada Mihalcea, Minlie Huang
| Challenge: | Existing benchmarks for Emotional Intelligence (EI) focus on emotion recognition, neglecting essential EI capabilities. |
| Approach: | They propose a benchmark that proposes a comprehensive definition for machine EI . they propose 400 hand-crafted questions in English and Chinese to evaluate EI. |
| Outcome: | The proposed benchmarks focus on emotion recognition, neglecting EI capabilities . they are constructed from existing datasets, which include frequent patterns and errors . the proposed benchmark includes questions in English and Chinese that require thorough reasoning and understanding . |
Guilt by Association: Emotion Intensities in Lexical Representations (2021.emnlp-main)
Copied to clipboard
| Challenge: | linguistic models have a higher correlation with human ground truth ratings than labeled data . word vectors have often been evaluated on standard word relatedness benchmarks . |
| Approach: | They propose to use unsupervised, supervised, and finally supervised methods to extract emotional associations from pretrained vectors and models. |
| Outcome: | The proposed method shows higher correlation with ground truth ratings than state-of-the-art lexicons based on labeled data. |
The Potential and Challenges of Evaluating Attitudes, Opinions, and Values in Large Language Models (2024.findings-emnlp)
Copied to clipboard
Bolei Ma, Xinpeng Wang, Tiancheng Hu, Anna-Carolina Haensch, Michael Hedderich, Barbara Plank, Frauke Kreuter
| Challenge: | Recent advances in Large Language Models have sparked interest in validating human-like cognitive-behavioral traits. |
| Approach: | They examine whether LLM outputs reflect human-like cognitive-behavioral traits . they find that measuring AOVs embedded within LLMs remains opaque . |
| Outcome: | The proposed model can be used to evaluate human-like cognitive-behavioral traits . the proposed model could be used in writing assistants and other applications . |
Cognitive Effects and Biases in Large Language Models (2026.eacl-tutorials)
Copied to clipboard
| Challenge: | This tutorial bridges psychology and NLP to clarify cognitive effects and biases in large language models. |
| Approach: | This tutorial bridges psychology and NLP to clarify cognitive effects and biases in large language models. |
| Outcome: | This tutorial bridges psychology and NLP to clarify cognitive effects and biases in large language models. |