| Challenge: | sarcasm and emotion are often used in conversational systems to generate the right response. |
| Approach: | They use a sarcastic expression dataset pre-annotated with 9 emotions to detect emotion . they identify and correct 343 incorrect emotion labels and label each sarkastic utterance with one of four sarcasm types. |
| Outcome: | The proposed model outperforms state-of-the-art sarcasm detection methods by using a multimodal sarcastic detection dataset. |
Similar Papers
Towards Multimodal Sarcasm Detection (An _Obviously_ Perfect Paper) (P19-1)
Copied to clipboard
Santiago Castro, Devamanyu Hazarika, Verónica Pérez-Rosas, Roger Zimmermann, Rada Mihalcea, Soujanya Poria
| Challenge: | sarcasm is often expressed through multiple verbal and non-verbal cues, such as a change of tone, overemphasis, drawn-out syllables, or a straight looking face. |
| Approach: | They propose to use multimodal cues to improve sarcasm detection using audiovisual utterances annotated with sarcasm labels to improve the accuracy. |
| Outcome: | The proposed dataset reduces the error rate of sarcasm detection by 12.9% . it is based on audiovisual utterances annotated with sarcasm labels . |
SarcNet: A Multilingual Multimodal Sarcasm Detection Dataset (2024.lrec-main)
Copied to clipboard
| Challenge: | Sarcasm is an implicit form of sarcasm, involving an intended meaning that contradicts the literal expression . human use conflict between factual information and a statement as cues to detect sarcasm . sarkasmatic analysis is challenging due to its implicit nature . |
| Approach: | They propose a multimodal sarcasm detection dataset that uses multiple modalities to detect sarcasm. |
| Outcome: | The proposed model improves on previous models based on a single label . human sarcasm cannot be detected using a unified label across multiple modalities . |
Sentiment and Emotion help Sarcasm? A Multi-task Learning Framework for Multi-Modal Sarcasm, Sentiment and Emotion Analysis (2020.acl-main)
Copied to clipboard
| Challenge: | Existing systems for sarcasm detection are limited by the use of sarcasm . sarasm is often used to convey thinly veiled disapproval humorously. |
| Approach: | They propose a multi-task deep learning framework to solve sarcasm problems simultaneously . they manually annotate a sarcsm dataset with sentiment and emotion classes . |
| Outcome: | The proposed framework is able to solve sarcasm, sentiment and emotion problems in a multi-modal conversational scenario. |
Leveraging Generative Large Language Models with Visual Instruction and Demonstration Retrieval for Multimodal Sarcasm Detection (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for multimodal sarcasm detection do not fully utilize cross-modal features, limiting their performance on in-domain datasets. |
| Approach: | They propose a multimodal sarcasm detection model with a designed instruction template and a demonstration retrieval module. |
| Outcome: | The proposed model outperforms existing methods on in-domain datasets and achieves state-of-the-art performance. |
Predict and Use: Harnessing Predicted Gaze to Improve Multimodal Sarcasm Detection (2023.emnlp-main)
Copied to clipboard
| Challenge: | sarcasm detection depends on content spoken, tonality, facial expressions, context, and personal traits like language proficiency and cognitive capabilities. |
| Approach: | They propose to use synthetic gaze data to improve sarcasm detection in conversational context . they collect gaze features for 20% of data instances and use them to predict gaze features . |
| Outcome: | The proposed model improves performance on a conversational dataset using gaze features . it achieves a gain of 6.6% points on the complete dataset with only predicted gaze features. |
Scale Is All You Need: Analyzing Modality Interaction and Speaker Intent Without Fine-Tuning (2026.eacl-srw)
Copied to clipboard
| Challenge: | Recent work on sarcasm and humor detection uses large multimodal Transformers, but they are computationally expensive and opaque. |
| Approach: | They propose a lightweight framework for multimodal sarcasm detection that combines frozen text, audio, and visual embeddings from pretrained encoders through compact fusion heads. |
| Outcome: | The proposed framework improves on the best unimodal baseline by combining text, audio, and visual embeddings from pretrained encoders with compact fusion heads. |
Action and Reaction Go Hand in Hand! a Multi-modal Dialogue Act Aided Sarcasm Identification (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing studies have shown that sarcasm is reflected by the intended meaning of the speaker's utterance. |
| Approach: | They propose to extend the MUStARD dataset to enclose dialogue acts for each dialogue . they propose a dialogue act-aided multi-modal transformer network for sarcasm identification model . |
| Outcome: | The proposed model improves performance in dialogue act-aided sarcasm identification compared to sardasmatic identification alone. |
Multi-View Incongruity Learning for Multimodal Sarcasm Detection (2025.coling-main)
Copied to clipboard
Diandian Guo, Cong Cao, Fangfang Yuan, Yanbing Liu, Guangjie Zeng, Xiaoyan Yu, Hao Peng, Philip S. Yu
| Challenge: | Existing methods for multimodal sarcasm detection rely on spurious correlations, demonstrating poor generalizability beyond training environments. |
| Approach: | They propose a method that integrates multimodal incongruities via contrastive learning for multimodal sarcasm detection by using three views to drive multi-view learning. |
| Outcome: | The proposed method outperforms existing methods on benchmark datasets and shows that it is more generalizable than existing methods. |
Multi-Modal Sarcasm Detection via Cross-Modal Graph Convolutional Network (2022.acl-long)
Copied to clipboard
| Challenge: | Existing studies on multimodal sarcasm detection using textual and visual information have been limited to text-only approaches. |
| Approach: | They propose to construct a cross-modal graph for each multi-modal instance to explicitly draw the ironic relations between textual and visual modalities. |
| Outcome: | The proposed model achieves state-of-the-art in multi-modal sarcasm detection. |
Breakthrough from Nuance and Inconsistency: Enhancing Multimodal Sarcasm Detection with Context-Aware Self-Attention Fusion and Word Weight Calculation. (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for sarcasm detection rely on feature concatenation to fuse different modalities or model inconsistencies among modalités. |
| Approach: | They propose to use Context-Aware Self-Attention Fusion to integrate local and momentary multimodal information into specific words to illustrate the inconsistencies between connotation and denotation. |
| Outcome: | The proposed method achieves an accuracy of 76.9 and an F1 score of 76.1 on the MUStARD dataset, surpassing the current state-of-the-art IWAN model by 1.7 and 1.6 respectively. |