| Challenge: | Existing tests on emotion datasets do not show whether language models understand emotions or exploit supperficial lexical cues. |
| Approach: | They propose to use two existing emotion datasets to evaluate whether language models make inferential decisions for emotion detection. |
| Outcome: | The proposed test sets evaluate language models on emotion datasets. |
Similar Papers
Learning and Evaluating Emotion Lexicons for 91 Languages (2020.acl-main)
Copied to clipboard
| Challenge: | Emotion lexicons describe the affective meaning of words but are limited in coverage for most languages. |
| Approach: | They propose a method for creating arbitrarily large emotion lexicons for any target language. |
| Outcome: | The proposed method exceeds human reliability for some languages and variables. |
What BERT Is Not: Lessons from a New Suite of Psycholinguistic Diagnostics for Language Models (2020.tacl-1)
Copied to clipboard
| Challenge: | Pretraining by language modeling has become popular but we have yet to understand what language models learn during that process. |
| Approach: | They propose diagnostics that ask questions about information used by language models for generating predictions in context. |
| Outcome: | The proposed diagnostics can be used to study the popular BERT model . they show that the model can distinguish good from bad completions, but struggles with inference and role-based event prediction. |
Evaluating Subjective Cognitive Appraisals of Emotions from Large Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work on automatic prediction of cognitive appraisals has focused on physiological aspects of emotions. |
| Approach: | They present a dataset that assesses 24 appraisal dimensions across 241 Reddit posts . they find that open-source models fail to automatically assess and explain cognitive appraisals . |
| Outcome: | The proposed dataset assesses 24 appraisal dimensions across 241 reddit posts. |
EmotionQueen: A Benchmark for Evaluating Empathy of Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing evaluations of emotional intelligence in large language models (LLMs) focus on basic sentiment analysis tasks, such as emotion recognition, which is not enough to evaluate LLMs’ overall emotional intelligence. |
| Approach: | They propose a framework for evaluating the emotional intelligence of large language models (LLMs) that includes four distinct tasks: Key Event Recognition, Mixed Event Recognition and Implicit Emotional Recognition. |
| Outcome: | The proposed framework includes four distinct tasks: Key Event Recognition, Mixed Event Recognition and Implicit Emotional Recognition. |
Guilt by Association: Emotion Intensities in Lexical Representations (2021.emnlp-main)
Copied to clipboard
| Challenge: | linguistic models have a higher correlation with human ground truth ratings than labeled data . word vectors have often been evaluated on standard word relatedness benchmarks . |
| Approach: | They propose to use unsupervised, supervised, and finally supervised methods to extract emotional associations from pretrained vectors and models. |
| Outcome: | The proposed method shows higher correlation with ground truth ratings than state-of-the-art lexicons based on labeled data. |
Characterizing and Evaluating Working Emotion Vocabularies in Multilingual Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Prior work evaluating emotion and affective understanding in large language models rely on predetermined label sets or focus on a singular evaluation task. |
| Approach: | They examine the ability of multilingual language models to predict any term used by an author to label their own feelings or emotions. |
| Outcome: | The proposed models perform poorly on three different tasks in English and Spanish. |
Do Emotions Influence Moral Judgment in Large Language Models? (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent systems enforce explicit ethical constraints, but moral judgment rarely involves such clear-cut prohibitions. |
| Approach: | They develop an emotion-induction pipeline that infuses emotion into moral situations and evaluate shifts in moral acceptability across datasets and LLMs. |
| Outcome: | The proposed pipeline can infuses emotion into moral situations and evaluate moral acceptability shifts across datasets and LLMs. |
Language Models (Mostly) Do Not Consider Emotion Triggers When Predicting Emotion (2024.naacl-short)
Copied to clipboard
| Challenge: | Existing work has sought to identify what triggers or causes a particular emotion, but the relationship between those triggers and the prediction of emotion detection models is little understood. |
| Approach: | They propose a dataset to evaluate the ability of large language models to identify emotion triggers . they compare features considered important for emotion prediction models to those considered less salient . |
| Outcome: | The proposed dataset compares large language models and fine-tuned models on social media posts . it shows that emotion triggers are not considered salient features for emotion prediction models . |
An Emotional Mess! Deciding on a Framework for Building a Dutch Emotion-Annotated Corpus (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing frameworks for emotion recognition are limited and do not allow for categorical versus dimensional oppositions. |
| Approach: | They propose to use the emotions joy, love, anger, sadness and fear as well as dimensional models to annotate texts from different domains and topics. |
| Outcome: | The proposed frameworks are well-suited to annotate texts from different domains and topics, but the connotation of the labels strongly depends on the origin of the texts. |
Cross-lingual Emotion Detection (2022.lrec-1)
Copied to clipboard
| Challenge: | Emotion detection is a useful tool for understanding human behavior, but constructing annotated datasets to train models can be expensive. |
| Approach: | They propose to use English as the source language with Arabic and Spanish as target languages to train models for emotion detection in a target language. |
| Outcome: | The proposed approaches surpass state-of-the-art models in Arabic and Spanish by 4% and 5% respectively. |