Towards Sentiment and Emotion aided Multi-modal Speech Act Classification in Twitter (2021.naacl-main)
Copied to clipboard
| Challenge: | Speech Act Classification determining the communicative intent of an utterance has been investigated widely over the years as a standalone task. |
| Approach: | They propose a multi-modal, emotion-TA dataset called EmoTA from open-source Twitter dataset and a Dyadic Attention Mechanism framework that integrates intra-modal and inter-modal attention to fuse multiple modalities. |
| Outcome: | The proposed framework boosts the performance of the primary task, i.e., TA classification (TAC), by benefitting from the two secondary tasks, namely, Sentiment and Emotion Analysis compared to its uni-modal and single task TAC variants. |
Similar Papers
Meta-Learning based Deferred Optimisation for Sentiment and Emotion aware Multi-modal Dialogue Act Classification (2022.aacl-main)
Copied to clipboard
| Challenge: | Empirically, we show that the optimisation of multi-modal DAC, SA and ER tasks produces better results compared to its different counterparts. |
| Approach: | They propose a dual attention mechanism that integrates sentiment tags into a multi-modal conversational framework that integrate modal attentions and multiple loss optimization. |
| Outcome: | The proposed framework integrates sentiment tags for each utterance and learns generalized features across multiple tasks. |
Towards Emotion-aided Multi-modal Dialogue Act Classification (2020.acl-main)
Copied to clipboard
| Challenge: | Considerable work on Dialogue Act Classification (DAC) has been done on textual inputs. |
| Approach: | They propose to use a multimodal Emotion aware Dialogue Act dataset to explore the role of multi-modality and emotion recognition in DAC. |
| Outcome: | The proposed dataset shows that multi-modality and emotion recognition improves DAC performance compared to uni-modal and single task DAC variants. |
Multi-task Learning for Multi-modal Emotion Recognition and Sentiment Analysis (N19-1)
Copied to clipboard
Md Shad Akhtar, Dushyant Chauhan, Deepanway Ghosal, Soujanya Poria, Asif Ekbal, Pushpak Bhattacharyya
| Challenge: | Existing frameworks for sentiment and emotion analysis are not efficient for inter-task learning. |
| Approach: | They propose a multi-task learning framework that performs sentiment and emotion analysis together. |
| Outcome: | The proposed framework improves on a CMU-MOSEI dataset for sentiment and emotion analysis. |
Sentiment and Emotion help Sarcasm? A Multi-task Learning Framework for Multi-Modal Sarcasm, Sentiment and Emotion Analysis (2020.acl-main)
Copied to clipboard
| Challenge: | Existing systems for sarcasm detection are limited by the use of sarcasm . sarasm is often used to convey thinly veiled disapproval humorously. |
| Approach: | They propose a multi-task deep learning framework to solve sarcasm problems simultaneously . they manually annotate a sarcsm dataset with sentiment and emotion classes . |
| Outcome: | The proposed framework is able to solve sarcasm, sentiment and emotion problems in a multi-modal conversational scenario. |
Self-supervised Cross-modal Pretraining for Speech Emotion Recognition and Sentiment Analysis (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to multimodal speech emotion recognition and sentiment analysis have not improved results due to their relatively simple fusion mechanisms and lack of proper cross-modal pretraining. |
| Approach: | They propose a deep-fused audio-text bi-modal transformer with carefully designed cross-modal fusion mechanism and stage-wise cross-mod pretraining scheme to facilitate cross-modulation. |
| Outcome: | The proposed method exceeds benchmarks on public IEMOCAP emotion and CMU-MOSEI sentiment datasets by a large margin. |
Multi-Channel Convolutional Neural Network for Twitter Emotion and Sentiment Recognition (N19-1)
Copied to clipboard
| Challenge: | Existing methods to analyze tweets are based on lexical features and a multi-channel convolutional neural architecture. |
| Approach: | They propose a neural network which can use different emotion and sentiment indicators such as hashtags, emoticons and emojis present in tweets to improve the performance of emotion and feelings identification. |
| Outcome: | The proposed model can use hashtags, emoticons and emojis present in tweets and improves emotion and sentiment identification. |
Context-aware Interactive Attention for Multi-modal Sentiment and Emotion Analysis (D19-1)
Copied to clipboard
| Challenge: | Multi-modal analysis is a field emerging in the fields of natural language processing, computer vision and speech processing . multimodal analysis uses a variety of information from multiple sources to build efficient systems . acoustic and visual information can provide better information for classification decisions . |
| Approach: | They propose a recurrent neural network based approach for multi-modal sentiment and emotion analysis . they employ a context-aware attention module to exploit the correspondence among neighboring utterances . |
| Outcome: | The proposed model learns inter-modal interaction among participating modalities through auto-encoder mechanism . it is compared with existing state-of-the-art models on five standard multi-modal affect analysis datasets . |
Multi-task dialog act and sentiment recognition on Mastodon (C18-1)
Copied to clipboard
| Challenge: | Social media are a gold mine for researchers in many domains and especially in natural language processing . license restrictions make it difficult to strictly reproduce research results on Twitter data . |
| Approach: | They propose to annotate a Twitter-like corpus from a decentralized social network with permissive licenses that are compatible with reproducible experiments. |
| Outcome: | The proposed method shows that transfer learning can be efficiently achieved between tasks. |
Multi-Task Learning Framework for Mining Crowd Intelligence towards Clinical Treatment (N18-2)
Copied to clipboard
| Challenge: | In recent past, social media has emerged as an active platform in the context of healthcare and medicine. |
| Approach: | They propose to use a novel adversarial learning approach to capture medical sentiments expressed in a medical blog to analyze the user's opinions on health-related issues. |
| Outcome: | The proposed framework can capture the user's opinions on health-related issues at a medical blog level. |
Understanding Emotions: A Dataset of Tweets to Study Interactions between Affect Categories (L18-1)
Copied to clipboard
| Challenge: | a new dataset is used to classify text into positive, negative, and neutral classes . a large amount of work on automatic detecting emotions from text has focused on classifying text into basic emotion categories . |
| Approach: | They use Twitter as the source of the textual data they annotate to find out which emotions often present together in tweets . |
| Outcome: | The proposed dataset is useful for training and testing supervised machine learning algorithms . it is based on the results of the SemEval-2018 task 1: Affect in Tweets . |