A Facial Expression-Aware Multimodal Multi-task Learning Framework for Emotion Recognition in Multi-party Conversations (2023.acl-long)
Copied to clipboard
| Challenge: | Recent studies have shown the importance of visual information in multi-party conversations due to the complexity of visual scenes. |
| Approach: | They propose a framework to extract face sequences as visual features from a real speaker's utterance and a pipeline method to extract the face sequence. |
| Outcome: | The proposed framework extracts face sequences of the real speaker of each utterance and improves emotion prediction on the MELD dataset. |
Similar Papers
MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations (P19-1)
Copied to clipboard
| Challenge: | Emotion recognition in conversations has gained popularity due to its potential applications. Until now, a large multimodal multi-party emotional conversational database containing more than two speakers per dialogue was missing. |
| Approach: | They propose to extend and enhance EmotionLines by combining 13,000 utterances from Friends dialogues with emotion and sentiment labels. |
| Outcome: | The proposed dataset contains about 13,000 utterances from 1,433 dialogues from the TV-series Friends. |
A Dual Contrastive Learning Framework for Enhanced Multimodal Conversational Emotion Recognition (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods struggle to capture emotion shifts due to label replication and fail to preserve positive independent modality contributions during fusion. |
| Approach: | They propose a Dual Contrastive Learning Framework that enhances existing MERC models without additional data. |
| Outcome: | The proposed framework outperforms existing models on two MERC benchmark datasets and shows that it reduces label dependence and enhances emotion-sensitive independent modality features. |
Multimodal Emotion Recognition in Conversations: A Survey of Methods, Trends, Challenges and Prospects (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Multimodal Emotion Recognition in Conversations (MERC) is a new way to enhance human-computer interaction. |
| Approach: | This survey offers a systematic overview of Multimodal Emotion Recognition in Conversations . it examines motivations, core tasks, representative methods, and evaluation strategies . |
| Outcome: | The survey examines the effectiveness of MERC and its evaluation strategies. |
MultiEMO: An Attention-Based Correlation-Aware Multimodal Fusion Framework for Emotion Recognition in Conversations (2023.acl-long)
Copied to clipboard
| Challenge: | Emotion Recognition in Conversations (ERC) is an increasingly popular task in the field of Natural Language Processing. |
| Approach: | They propose a framework that captures cross-modal mapping relationships across modalities . they propose 'multiemotion-aware' framework that integrates multimodal cues into the model . |
| Outcome: | The proposed framework outperforms state-of-the-art models in all emotion categories on two benchmark datasets. |
A Sentiment and Emotion Aware Multimodal Multiparty Humor Recognition in Multilingual Conversational Setting (2022.coling-1)
Copied to clipboard
| Challenge: | Humor is an essential aspect of daily conversation, and people try to provoke humor in their talks. |
| Approach: | They propose a multitask framework that annotates Hindi utterances with sentiment and emotion classes. |
| Outcome: | The proposed framework improves on the recently released Hindi Humor dataset . it takes sentiment and emotion into account to understand humor . |
Self-adaptive Context and Modal-interaction Modeling For Multimodal Emotion Recognition (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to predict emotion label for a given utterance lack modeling of diverse dependency ranges and inconsistent treatment of contribution for various modalities. |
| Approach: | They propose a multimodal emotion recognition in conversation task that uses context and multiple modalities to predict emotion label for a given utterance. |
| Outcome: | The proposed method outperforms the state-of-the-art methods on three multimodal datasets. |
UniMSE: Towards Unified Multimodal Sentiment Analysis and Emotion Recognition (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies study sentiment and emotion separately and do not fully exploit the complementary knowledge behind the two. |
| Approach: | They propose a multimodal sentiment knowledge-sharing framework that unifies MSA and ERC tasks from features, labels, and models. |
| Outcome: | The proposed framework achieves consistent improvements on four public benchmark datasets on MOSI, MOSEI, MELD, and IEMOCAP. |
Multi-task Learning for Multi-modal Emotion Recognition and Sentiment Analysis (N19-1)
Copied to clipboard
Md Shad Akhtar, Dushyant Chauhan, Deepanway Ghosal, Soujanya Poria, Asif Ekbal, Pushpak Bhattacharyya
| Challenge: | Existing frameworks for sentiment and emotion analysis are not efficient for inter-task learning. |
| Approach: | They propose a multi-task learning framework that performs sentiment and emotion analysis together. |
| Outcome: | The proposed framework improves on a CMU-MOSEI dataset for sentiment and emotion analysis. |
EmotionTalk: An Interactive Chinese Multimodal Emotion Dataset With Rich Annotations (2026.findings-acl)
Copied to clipboard
Haoqin Sun, Jinghua Zhao, Xuechen Wang, Shiwan Zhao, Jiaming Zhou, Hui Wang, Xi Yang, Yequan Wang, Yonghua Lin
| Challenge: | Existing datasets face issues such as low quality, limited scale, and incomplete modalities, hindering model performance. |
| Approach: | They propose to use Chinese multimodal datasets to capture authentic emotional interplay from 19 professional actors. |
| Outcome: | The EmotionTalk dataset spans 23.6 hours of dyadic conversations across diverse scenarios. |
COMMA-DEER: COmmon-sense Aware Multimodal Multitask Approach for Detection of Emotion and Emotional Reasoning in Conversations (2022.coling-1)
Copied to clipboard
| Challenge: | Mental health is a critical component of the United Nations’ Sustainable Development Goals (SDGs), particularly Goal 3 which aims to provide “good health and well-being”. |
| Approach: | They propose a task of detecting emotional reasoning and accompanying emotions in conversations that is manually annotated at the utterance level. |
| Outcome: | The proposed model achieves 6% accuracy and 4.62% accuracy on the emotion detection task and 3.56% accuracy, and 3.31% F1 on the ER detection task, compared to the existing state-of-the-art model. |