Multimodal Emotion Recognition in Conversations: A Survey of Methods, Trends, Challenges and Prospects (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Multimodal Emotion Recognition in Conversations (MERC) is a new way to enhance human-computer interaction. |
| Approach: | This survey offers a systematic overview of Multimodal Emotion Recognition in Conversations . it examines motivations, core tasks, representative methods, and evaluation strategies . |
| Outcome: | The survey examines the effectiveness of MERC and its evaluation strategies. |
Similar Papers
MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations (P19-1)
Copied to clipboard
| Challenge: | Emotion recognition in conversations has gained popularity due to its potential applications. Until now, a large multimodal multi-party emotional conversational database containing more than two speakers per dialogue was missing. |
| Approach: | They propose to extend and enhance EmotionLines by combining 13,000 utterances from Friends dialogues with emotion and sentiment labels. |
| Outcome: | The proposed dataset contains about 13,000 utterances from 1,433 dialogues from the TV-series Friends. |
Multiple Knowledge-Enhanced Interactive Graph Network for Multimodal Conversational Emotion Recognition (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Multimodal Emotion Recognition in Conversations models struggle due to lack of Common Sense Knowledge (CSK). |
| Approach: | They propose a multimodal approach to integrate multiple knowledge into the edge representations by integrating textual and visual CSK. |
| Outcome: | The proposed model outperforms state-of-the-art methods on two popular datasets. |
Emotion-Wheel-Guided Audio-Referred Text Representation for Multimodal Emotion Recognition in Conversation (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for Emotion Recognition in Conversation ignore their distinct communicative roles and information capacities and apply uniform penalties regardless of affective proximity. |
| Approach: | They propose a modality-aware fusion strategy capturing linguistic features from text as the primary source and audio as a complementary component. |
| Outcome: | The proposed method captures linguistic features from text as the primary source and audio as a complementary component and supervised contrastive loss to encode emotional proximity based on Russell’s circumplex model. |
A Dual Contrastive Learning Framework for Enhanced Multimodal Conversational Emotion Recognition (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods struggle to capture emotion shifts due to label replication and fail to preserve positive independent modality contributions during fusion. |
| Approach: | They propose a Dual Contrastive Learning Framework that enhances existing MERC models without additional data. |
| Outcome: | The proposed framework outperforms existing models on two MERC benchmark datasets and shows that it reduces label dependence and enhances emotion-sensitive independent modality features. |
MultiEMO: An Attention-Based Correlation-Aware Multimodal Fusion Framework for Emotion Recognition in Conversations (2023.acl-long)
Copied to clipboard
| Challenge: | Emotion Recognition in Conversations (ERC) is an increasingly popular task in the field of Natural Language Processing. |
| Approach: | They propose a framework that captures cross-modal mapping relationships across modalities . they propose 'multiemotion-aware' framework that integrates multimodal cues into the model . |
| Outcome: | The proposed framework outperforms state-of-the-art models in all emotion categories on two benchmark datasets. |
UniMSE: Towards Unified Multimodal Sentiment Analysis and Emotion Recognition (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies study sentiment and emotion separately and do not fully exploit the complementary knowledge behind the two. |
| Approach: | They propose a multimodal sentiment knowledge-sharing framework that unifies MSA and ERC tasks from features, labels, and models. |
| Outcome: | The proposed framework achieves consistent improvements on four public benchmark datasets on MOSI, MOSEI, MELD, and IEMOCAP. |
MEISD: A Multimodal Multi-Label Emotion, Intensity and Sentiment Dialogue Dataset for Emotion Recognition and Sentiment Analysis in Conversations (2020.coling-main)
Copied to clipboard
| Challenge: | Emotion and sentiment classification in dialogues has gained popularity in recent times . a number of datasets are imbalanced in representing different emotions and consist of an only single emotion. |
| Approach: | They propose to use a dataset to analyze emotions and sentiments in dialogues . they use text, audio and video to identify the correct emotions with the appropriate intensity and sentiment in an utterance of a dialogue . |
| Outcome: | The proposed datasets are balanced in representing different emotions and consist of only one emotion. |
EmotionTalk: An Interactive Chinese Multimodal Emotion Dataset With Rich Annotations (2026.findings-acl)
Copied to clipboard
Haoqin Sun, Jinghua Zhao, Xuechen Wang, Shiwan Zhao, Jiaming Zhou, Hui Wang, Xi Yang, Yequan Wang, Yonghua Lin
| Challenge: | Existing datasets face issues such as low quality, limited scale, and incomplete modalities, hindering model performance. |
| Approach: | They propose to use Chinese multimodal datasets to capture authentic emotional interplay from 19 professional actors. |
| Outcome: | The EmotionTalk dataset spans 23.6 hours of dyadic conversations across diverse scenarios. |
Emotion Recognition in Multi-Speaker Conversations through Speaker Identification, Knowledge Distillation, and Hierarchical Fusion (2026.findings-eacl)
Copied to clipboard
| Challenge: | Emotion recognition in multi-speaker conversations faces significant challenges due to speaker ambiguity and severe class imbalance. |
| Approach: | They propose a speaker identification module that leverages audio-visual synchronization to accurately identify the active speaker and hierarchical attention fusion with composite loss functions to handle class imbalance. |
| Outcome: | The proposed framework achieves 67.75% and 72.44% weighted F1 scores on MELD and IEMOCAP datasets, with notable improvements on minority emotion classes. |
EmotionLines: An Emotion Corpus of Multi-Party Conversations (L18-1)
Copied to clipboard
| Challenge: | Emotion is a critical characteristic to distinguish people from machines. |
| Approach: | They propose a dataset with emotions labeling on all utterances in each dialogue . they use Friends TV scripts and Facebook messenger dialogues to collect the data . |
| Outcome: | The proposed dataset is the first with emotions labeling on all utterances in each dialogue based on their textual content. |