Labeling Comic Mischief Content in Online Videos with a Multimodal Hierarchical-Cross-Attention Model (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing systems for detecting questionable content in online media are limited by age, life experiences, socio-cultural values, and cognitive skills. |
| Approach: | They propose a multimodal system for comic mischief detection using video, text, and audio. |
| Outcome: | The proposed system improves existing models and baselines for comic mischief detection and its type classification. |
Similar Papers
Thesis Proposal: An Explainable Multimodal Framework for Detecting Harmful Content in Code-Switched Children’s Media (2026.acl-srw)
Copied to clipboard
| Challenge: | Current content moderation systems fail to protect children from harmful content, especially in under-resourced, code-switched settings. |
| Approach: | They propose to integrate a fine-tuned classifier with an LLM-powered module that synthesizes the classifier’s internal evidential signals to generate faithful, human-readable rationales for each decision. |
| Outcome: | The proposed framework integrates a fine-tuned classifier for accurate, scalable detection with an LLM-powered module that synthesizes the classifier’s internal evidential signals to generate faithful, human-readable rationales for each decision. |
A Survey on Multimodal Disinformation Detection (2022.coling-1)
Copied to clipboard
Firoj Alam, Stefano Cresci, Tanmoy Chakraborty, Fabrizio Silvestri, Dimiter Dimitrov, Giovanni Da San Martino, Shaden Shaar, Hamed Firooz, Preslav Nakov
| Challenge: | Recent years have witnessed the proliferation of offensive content online such as fake news, propaganda, misinformation, and disinformation. |
| Approach: | They propose to tackle online multimodal offensive content using different modalities and combinations thereof. |
| Outcome: | The proposed approach combines factuality and harmfulness in a framework that can be used for multiple modalities and combinations of modality. |
Humor in Pixels: Benchmarking Large Multimodal Models Understanding of Online Comics (2025.findings-emnlp)
Copied to clipboard
| Challenge: | PixelHumor is a benchmark dataset of 2,800 annotated multi-panel comics designed to evaluate LMMs’ ability to interpret multimodal humor and recognize narrative sequences. |
| Approach: | PixelHumor is a benchmark dataset of 2,800 annotated multi-panel comics designed to evaluate LMMs’ ability to interpret multimodal humor and recognize narrative sequences. |
| Outcome: | Experiments with state-of-the-art LMMs reveal that top models achieve only 61% accuracy in panel sequencing, far below human performance. |
Visual Attention Model for Name Tagging in Multimodal Social Media (P18-1)
Copied to clipboard
| Challenge: | Name tagging is a key task for language understanding, but is often limited by the short textual components. |
| Approach: | They propose a novel model architecture based on visual attention that outperforms other methods . they use multimodal datasets to analyze the name tagging task on social media . |
| Outcome: | The proposed model outperforms existing methods and significantly outperformed existing methods. |
UR-FUNNY: A Multimodal Language Dataset for Understanding Humor (D19-1)
Copied to clipboard
Md Kamrul Hasan, Wasifur Rahman, AmirAli Bagher Zadeh, Jianyuan Zhong, Md Iftekhar Tanveer, Louis-Philippe Morency, Mohammed (Ehsan) Hoque
| Challenge: | Humor is a unique and creative communicative behavior often displayed during social interactions. |
| Approach: | They present a dataset that allows to model multimodal language used in expressing humor using text, visual and acoustic communication. |
| Outcome: | The proposed framework opens the door to understanding multimodal language used in expressing humor. |
Improving Multimodal Classification of Social Media Posts by Leveraging Image-Text Auxiliary Tasks (2024.findings-eacl)
Copied to clipboard
| Challenge: | Prior work on multimodal content classification has not addressed these challenges. |
| Approach: | They propose to use two auxiliary tasks to fine-tune multimodal models to address hidden cross-modal semantics and weak image-text relationships when modeling text and images. |
| Outcome: | The proposed model improves by up to 2.6 F1 score across five diverse social media datasets. |
Not all Fake News is Written: A Dataset and Analysis of Misleading Video Headlines (2023.emnlp-main)
Copied to clipboard
| Challenge: | Social media platforms are used by half of U.S. adults for everyday news consumption. |
| Approach: | They propose to analyze video headlines and whether annotators believe the headline is representative of the video’s contents. |
| Outcome: | The proposed dataset analyzes video headlines and explains why annotators view a video as misleading. |
ToxVidLM: A Multimodal Framework for Toxicity Detection in Code-Mixed Videos (2024.findings-acl)
Copied to clipboard
| Challenge: | Using a dataset of 931 videos with 4021 code-mixed Hindi-English utterances, we find that video content with multiple modalities is more accurate and more accurate than textual content. |
| Approach: | They propose to use a dataset to analyze toxic content in video content in non-English languages by leveraging language models. |
| Outcome: | The proposed framework achieves an Accuracy and Weighted F1 score of 94.29% and 94.35% for the first time in its class. |
Multi-Modal Sarcasm Detection in Twitter with Hierarchical Fusion Model (P19-1)
Copied to clipboard
| Challenge: | Existing methods to detect sarcasm focus on text, but they are insufficient for multi-modal messages. |
| Approach: | They propose a multi-modal hierarchical sarcasm detection model for tweets consisting of texts and images in Twitter. |
| Outcome: | The proposed model is able to detect sarcasm on twitter using three modalities . the proposed model can be used in customer service, opinion mining and harassment detection . |
TriSPrompt: A Hierarchical Soft Prompt Model for Multimodal Rumor Detection with Incomplete Modalities (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing multimodal rumor detection methods focus on learning joint modality representations from complete multimodal training data, rendering them ineffective in addressing the common occurrence of missing modalities in real-world scenarios. |
| Approach: | They propose a hierarchical soft prompt model TriSPrompt which integrates three types of prompts to effectively detect rumors in incomplete multimodal data. |
| Outcome: | The proposed model achieves an accuracy gain of over 13% compared to state-of-the-art models. |