Papers with IEMOCAP
Emo Pillars: Knowledge Distillation to Support Fine-Grained Context-Aware and Context-Less Emotion Classification (2025.findings-acl)
Copied to clipboard
| Challenge: | a recent study shows that sentiment analysis datasets lack context in which an opinion was expressed and are limited by a few emotion categories. |
| Approach: | They propose to ground an LLM-based model into a corpus of narratives to generate stories-character-centered utterances with unique contexts over 28 emotion classes. |
| Outcome: | The proposed model generates non-repetitive story-character-centered utterances with unique contexts over 28 emotion classes. |
Missing Modality meets Meta Sampling (M3S): An Efficient Universal Approach for Multimodal Sentiment Analysis with Missing Modality (2022.aacl-main)
Copied to clipboard
| Challenge: | Existing methods to address missing modalities often assume a particular modality is completely missing due to recording or transmission error. |
| Approach: | They propose a missing modality-based meta-sampling approach for multimodal sentiment analysis with missing modalities . they conduct experiments on IEMOCAP, SIMS and CMU-MOSI datasets . |
| Outcome: | The proposed method significantly improves on existing models with a mixture of missing modalities. |
Extreme Fine-tuning: A Novel and Fast Fine-tuning Approach for Text Classification (2024.eacl-short)
Copied to clipboard
| Challenge: | Existing methods for fine-tuning pre-trained models require massive computational resources and time. |
| Approach: | They propose a novel approach for fine-tuning a pre-trained model using backpropagation and an iterative extreme learning machine for training a classifier. |
| Outcome: | The proposed approach outperforms state-of-the-art approaches in training-time measurement and performance with comparable model performance. |
HiGRU: Hierarchical Gated Recurrent Units for Utterance-Level Emotion Recognition (N19-1)
Copied to clipboard
| Challenge: | Using textual features, our proposed HiGRU models achieve at least 8.7%, 7.5%, 6.0% improvement over the state-of-the-art methods on each dataset. |
| Approach: | They propose a hierarchical gated recurrent unit framework to model word-level inputs and an upper-level GRU to capture contexts of utterance-level embeddings. |
| Outcome: | The proposed framework achieves 8.7%, 7.5%, 6.0% improvement over state-of-the-art methods on three datasets. |
CMTD: Cognitive Modeling with Traits and Distortions for Multimodal Emotion Recognition in Conversations (2026.findings-acl)
Copied to clipboard
| Challenge: | Experimental results show that traits temper negativity bias from distortions, and cognitive modeling with psychological, visual, and acoustic information can improve the performance of MERC. |
| Approach: | They propose a framework for multimodal emotion recognition in conversations that takes advantage of stable personality traits, dynamic cognitive distortions, visual and acoustic features of interlocutors to enhance the emotional intelligence of LLMs. |
| Outcome: | Experimental results show that traits temper negativity bias from distortions, and cognitive modeling with psychological, visual, and acoustic information can improve the performance of MERC. |
Aligning Paralinguistic Understanding and Generation in Speech LLMs via Multi-Task Reinforcement Learning (2026.eacl-industry)
Copied to clipboard
Minseok Kim, Jingxiang Chen, Seong-Gyun Leem, Yin Huang, Rashi Rungta, Zhicheng Ouyang, Haibin Wu, Surya Teja Appini, Ankur Bansal, Yang Bai, Yue Liu, Florian Metze, Ahmed A Aly, Anuj Kumar, Ariya Rastrow, Zhaojiang Lin
| Challenge: | Using paralinguistic cues is challenging for speech large language models, authors say . limited training data, annotation difficulty, and models exploiting lexical shortcuts are challenges . a recent study shows that modeling paralinguistic reasoning with multitask RL improves paralinguistics understanding . |
| Approach: | They propose multi-task reinforcement learning with chain-of-thought prompting that elicits explicit affective reasoning. |
| Outcome: | The proposed model improves paralinguistics understanding over baselines and strong proprietary models by 8-12% on Expresso, IEMOCAP, and RAVDESS. |
DialogueEIN: Emotion Interaction Network for Dialogue Affective Analysis (2022.coling-1)
Copied to clipboard
| Challenge: | Emotion Recognition in Conversation (ERC) has attracted increasing research attention in recent years. |
| Approach: | They propose to model the emotional interactions between speakers to simulate the emotional inertia, emotional stimulus, global and local emotional evolution in dialogues. |
| Outcome: | The proposed model can achieve superior performance compared to state-of-the-art methods on four ERC benchmark datasets, IEMOCAP, MELD, EmoryNLP and DailyDialog. |
EDA: Enriching Emotional Dialogue Acts using an Ensemble of Neural Annotators (2020.lrec-1)
Copied to clipboard
| Challenge: | Emotion recognition helps to build natural dialogue systems. |
| Approach: | They propose to use a recurrent neural model to annotate emotion corpora with dialogue act labels and an ensemble annotator to extract the final dialogue act label. |
| Outcome: | The proposed model annotates two accessible multi-modal emotion corpora with and without context and extracts the final dialogue act label. |
ECERC: Evidence-Cause Attention Network for Multi-Modal Emotion Recognition in Conversation (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for multi-modal emotion recognition in isolated utterances do not capture emotional causes, including emotional contagion, influences from others, and self-referenced or externally introduced events. |
| Approach: | They propose a multi-modal conversational emotion recognition system that integrates emotional evidence with contextual causes through five stages. |
| Outcome: | The proposed method achieves competitive performance on two widely used benchmark datasets, IEMOCAP and MELD. |
Handling Ambiguity in Emotion: From Out-of-Domain Detection to Distribution Estimation (2024.acl-long)
Copied to clipboard
| Challenge: | Experimental results show that incorporating utterances without majority-agreed labels into an additional class reduces the classification performance of the other emotion classes. |
| Approach: | They propose to combine utterances without majority-agreed labels into an additional class . they propose to quantify uncertainty in emotion classification using evidential deep learning . |
| Outcome: | The proposed method retains classification accuracy while effectively detects ambiguous emotion expressions. |
Beyond Silent Letters: Amplifying LLMs in Emotion Recognition with Vocal Nuances (2025.findings-naacl)
Copied to clipboard
| Challenge: | Recent studies have demonstrated that Large Language Models possess a form of emotional intelligence, capable of interpreting emotional stimuli in text. |
| Approach: | They propose a method that translates speech characteristics into natural language descriptions and integrates them into LLMs to perform multimodal emotion analysis via text prompts. |
| Outcome: | The proposed method outperforms baseline models that require structural modifications on two datasets showing significant improvements in emotion recognition accuracy. |
Multi-Condition Guided Diffusion Network for Multimodal Emotion Recognition in Conversation (2025.findings-naacl)
Copied to clipboard
| Challenge: | Current research emphasizes contextual factors, the speaker’s influence, and extracting complementary information across different modalities. |
| Approach: | They propose a diffusion-based approach to address the challenges posed by redundant information and redundant information at the semantic level while robustly capturing shared semantics. |
| Outcome: | The proposed model outperforms existing state-of-the-art models on two multimodal datasets and is generalizable and effective. |
A Dual Contrastive Learning Framework for Enhanced Multimodal Conversational Emotion Recognition (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods struggle to capture emotion shifts due to label replication and fail to preserve positive independent modality contributions during fusion. |
| Approach: | They propose a Dual Contrastive Learning Framework that enhances existing MERC models without additional data. |
| Outcome: | The proposed framework outperforms existing models on two MERC benchmark datasets and shows that it reduces label dependence and enhances emotion-sensitive independent modality features. |
COGMEN: COntextualized GNN based Multimodal Emotion recognitioN (2022.naacl-main)
Copied to clipboard
| Challenge: | During a conversation, a person’s emotions are influenced by the other speaker’s utterances and their own emotional state over the utterrances. |
| Approach: | They propose a Graph Neural Network based Multi-modal Emotion recognitioN system that leverages local and global information in a conversation. |
| Outcome: | The proposed system gives state-of-the-art results on IEMOCAP and MOSEI datasets and detailed ablation experiments show the importance of modeling information at both levels. |
HiTrans: A Transformer-Based Context- and Speaker-Sensitive Model for Emotion Detection in Conversations (2020.coling-main)
Copied to clipboard
| Challenge: | Emotion detection in conversations is to detect the emotion for each utterance in conversations that have multiple speakers. |
| Approach: | They propose a transformer-based context- and speaker-sensitive model for EDC . they utilize a low-level transformer to generate local utterance representations . |
| Outcome: | The proposed model outperforms state-of-the-art models on three benchmark datasets. |
Curriculum Learning Meets Directed Acyclic Graph for Multimodal Emotion Recognition (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing models for multimodal Emotion Recognition in conversation (ERC) use text as the main modality for emotion recognition. |
| Approach: | They propose a Directed Acyclic Graph (DAG) approach that integrates textual, acoustic, and visual features within a unified framework. |
| Outcome: | The proposed model outperforms baseline models on the IEMOCAP and MELD datasets. |
Self-adaptive Context and Modal-interaction Modeling For Multimodal Emotion Recognition (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to predict emotion label for a given utterance lack modeling of diverse dependency ranges and inconsistent treatment of contribution for various modalities. |
| Approach: | They propose a multimodal emotion recognition in conversation task that uses context and multiple modalities to predict emotion label for a given utterance. |
| Outcome: | The proposed method outperforms the state-of-the-art methods on three multimodal datasets. |
MMGCN: Multimodal Fusion via Deep Graph Convolution Network for Emotion Recognition in Conversation (2021.acl-long)
Copied to clipboard
| Challenge: | Emotion recognition in conversation is a crucial component in affective dialogue systems, which helps the system understand users’ emotions and generate empathetic responses. |
| Approach: | They propose a multimodal fused graph convolutional network model which leverages multimodal dependencies and speaker information to model inter-speaker and intra-speech dependency. |
| Outcome: | The proposed model outperforms other SOTA methods on two public benchmark datasets, IEMOCAP and MELD. |
EmoTrans: Emotional Transition-based Model for Emotion Recognition in Conversation (2024.lrec-main)
Copied to clipboard
| Challenge: | Emotions are causally transmitted among communication participants, facilitating comprehension of intricate changes in emotional states during the conversation. |
| Approach: | They propose an Emotional Transition-based Emotion Recognizer that captures ET features in an emotional conversation by concatenating the most recent utterances with their corresponding speakers. |
| Outcome: | The proposed model is sensitive to emotions and captures ET features in the sample. |
UniMSE: Towards Unified Multimodal Sentiment Analysis and Emotion Recognition (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies study sentiment and emotion separately and do not fully exploit the complementary knowledge behind the two. |
| Approach: | They propose a multimodal sentiment knowledge-sharing framework that unifies MSA and ERC tasks from features, labels, and models. |
| Outcome: | The proposed framework achieves consistent improvements on four public benchmark datasets on MOSI, MOSEI, MELD, and IEMOCAP. |
ResFormer: All-Time Reservoir Memory for Long Sequence Classification (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing models with quadratic time and memory complexity restrict input length . however, analyzing extensive sequential contexts is challenging . |
| Approach: | They propose a neural network architecture that captures contextual dependencies in linear time and a nonlinear readout to model short-term dependencies within sentences. |
| Outcome: | The proposed model outperforms baseline models on EmoryNLP datasets and on IEMOCAP and MultiWOZ datasets. |
MMDAG: Multimodal Directed Acyclic Graph Network for Emotion Recognition in Conversation (2022.lrec-1)
Copied to clipboard
| Challenge: | Emotion recognition in conversation is important for an empathetic dialogue system to understand the user’s emotion and then generate appropriate emotional responses. |
| Approach: | They propose to use multimodal directed acyclic graphs to integrate multimodal information and contextual information into a DAG architecture to exploit multimodal contexts. |
| Outcome: | Comparative studies on IEMOCAP and MELD show that the proposed model outperforms state-of-the-art models. |
The MERSA Dataset and a Transformer-Based Approach for Speech Emotion Recognition (2024.acl-long)
Copied to clipboard
| Challenge: | Existing models for speech emotion recognition lack a comprehensive dataset to design accurate models. |
| Approach: | They propose to use a multimodal dataset to build a model that integrates pre-trained wav2vec 2.0 and BERT to learn hidden representations from fused representations of speech and text. |
| Outcome: | The proposed model predicts emotions on dimensions of arousal, valence, and dominance . it achieved competitive results on the MSP-PODCAST dataset . |
QAP: A Quantum-Inspired Adaptive-Priority-Learning Model for Multimodal Emotion Recognition (2023.findings-acl)
Copied to clipboard
| Challenge: | Experimental results show that multimodal emotion recognition is a state-of-the-art technique . textual, visual and acoustic modalities are involved in multimodal video emotion recognition . |
| Approach: | They propose a quantum-inspired adaptive-priority-learning model to address the challenges . they use quantum state to model modal features and Q-attention to integrate three modalities . |
| Outcome: | Experimental results show that QAP improves on previous models. |
Integrating Representation Subspace Mapping with Unimodal Auxiliary Loss for Attention-based Multimodal Emotion Recognition (2024.lrec-main)
Copied to clipboard
Xulong Du, Xingnan Zhang, Dandan Wang, Yingying Xu, Zhiyuan Wu, Shiqing Zhang, Xiaoming Zhao, Jun Yu, Liangliang Lou
| Challenge: | Existing methods to identify emotions rely on a large modality gap in their representations . |
| Approach: | They propose a representation subspace mapping module that maps each modality into two distinct subspaces and a cross-modality attention module that leverages auxiliary loss to remove the noise unrelated to emotion classification. |
| Outcome: | The proposed approach achieves superior performance to state-of-the-art MER methods on the IEMOCAP and MSP-Improv datasets. |
CaM-HG: Causal-Enhanced MoE and Hypergraphs Network for Incomplete Multimodal Emotion Recognition in Conversations (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods that focus on statistical reconstruction often fail to bridge these gaps, effectively leaving semantic holes. |
| Approach: | They propose a Causal-Enhanced Mixture-of-Experts and Hypergraph Network to bridge missing features . they use experts to synthesize missing features that are realistic and causally consistent . |
| Outcome: | The proposed model synthesizes missing features that are realistic and causally consistent . it surpasses benchmarks on IEMOCAP, CMU-MOSI, and CMU MOSEI by 1.43% and 1.25% . |
Conversation Understanding using Relational Temporal Graph Neural Networks with Auxiliary Cross-Modality Interaction (2023.emnlp-main)
Copied to clipboard
| Challenge: | Emotion recognition is a crucial task for human conversation understanding . multimodal data, e.g., language, voice, and facial expressions, add complexity to the task. |
| Approach: | They propose a relational temporal Graph Neural Network with Auxiliary Cross-Modality Interaction framework that captures conversation-level cross-modality interactions and utterance-level temporal dependencies with modality-specific manner for conversation understanding. |
| Outcome: | The proposed framework captures conversation-level cross-modality interactions and utterance-level temporal dependencies with the modality-specific manner for conversation understanding. |
CoE: A Clue of Emotion Framework for Emotion Recognition in Conversations (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are limited in interpreting complex conversational streams. |
| Approach: | They propose a Clue of Emotion framework which integrates key conversational clues to enhance the ERC task. |
| Outcome: | The proposed framework outperforms EmoryNLP, MELD, and IEMOCAP in the role-playing, speaker identification, and emotion reasoning tasks. |
Emotion-Wheel-Guided Audio-Referred Text Representation for Multimodal Emotion Recognition in Conversation (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for Emotion Recognition in Conversation ignore their distinct communicative roles and information capacities and apply uniform penalties regardless of affective proximity. |
| Approach: | They propose a modality-aware fusion strategy capturing linguistic features from text as the primary source and audio as a complementary component. |
| Outcome: | The proposed method captures linguistic features from text as the primary source and audio as a complementary component and supervised contrastive loss to encode emotional proximity based on Russell’s circumplex model. |