MOMENTA: A Multimodal Framework for Detecting Harmful Memes and Their Targets (2021.findings-emnlp)
Copied to clipboard
Shraman Pramanick, Shivam Sharma, Dimitar Dimitrov, Md. Shad Akhtar, Preslav Nakov, Tanmoy Chakraborty
| Challenge: | a growing number of harmful memes are being used for trolling, cyberbullying and abuse . a new approach to detect harmful meme images and texts is emerging . |
| Approach: | They propose a multimodal deep neural network that detects harmful memes . they extend the recently released HarMeme dataset with additional memes and a new topic . |
| Outcome: | The proposed framework outperforms rival methods in detecting harmful memes and their target social entities. |
Similar Papers
Detecting Harmful Memes and Their Targets (2021.findings-acl)
Copied to clipboard
Shraman Pramanick, Dimitar Dimitrov, Rituparna Mukherjee, Shivam Sharma, Md. Shad Akhtar, Preslav Nakov, Tanmoy Chakraborty
| Challenge: | a growing body of research on meme analysis has focused on detecting harmful memes and their social entities . a meme is a form of content that is often harmless and designed to look funny . but its multimodal nature and camouflaged semantics make its analysis challenging . |
| Approach: | They propose to use multimodal models to detect harmful memes and identify social entities that harmful meme targets. |
| Outcome: | The proposed model can detect harmful memes and the social entities they target . the proposed model lacks the appropriate contexts and is poorly validated . |
A Multimodal Framework to Detect Target Aware Aggression in Memes (2024.eacl-long)
Copied to clipboard
| Challenge: | Recent research on memes’ detrimental facets is skewed towards high-resource languages, such as Bengali. |
| Approach: | They propose a dataset MIMOSA that annotates annotated memes across five aggression target categories in Bengali and propose 'Multimodal Attentive Fusion' to detect aggression targets. |
| Outcome: | The proposed method outperforms state-of-the-art methods in Bengali and in low-resource languages. |
DISARM: Detecting the Victims Targeted by Harmful Memes (2022.findings-naacl)
Copied to clipboard
| Challenge: | DISARM is a framework that uses named-entity recognition and person identification to detect all entities a meme is referring to and then incorporates a novel contextualized deep neural network to classify whether the meme intends to harm these entities. |
| Approach: | They propose a framework that uses named-entity recognition and person identification to detect all entities a meme is referring to and incorporates a novel contextualized deep neural network to classify whether the meme intends to harm them. |
| Outcome: | The proposed framework outperforms 10 unimodal and multimodal systems and reduces error rate of harmful target identification by 9 % absolute over baseline systems. |
Beneath the Surface: Unveiling Harmful Memes with Multimodal Reasoning Distilled from Large Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for harmful meme detection ignore in-depth cognition of meme text and image . authors propose a framework for learning reasonable thoughts from LLMs for better multimodal fusion . |
| Approach: | They propose to use large language models to learn reasonable thoughts from LLMs for better multimodal fusion and lightweight fine-tuning. |
| Outcome: | The proposed approach achieves superior performance than state-of-the-art methods on the harmful meme detection task. |
Deciphering Hate: Identifying Hateful Memes and Their Targets (2024.acl-long)
Copied to clipboard
| Challenge: | a growing body of research has focused on the negative aspects of memes in high-resource languages like Bengali . a new dataset for Bengali hateful memes is designed to detect their targeted entities . |
| Approach: | They propose a multimodal dataset that analyzes the modality of memes and compares them with other datasets. |
| Outcome: | The proposed dataset outperforms state-of-the-art datasets on Bengali hateful memes . the proposed dataset is generalizable on other low-resource hateful memes datasets compared with baselines based on the proposed model . |
Characterizing the Entities in Harmful Memes: Who is the Hero, the Villain, the Victim? (2023.eacl-main)
Copied to clipboard
Shivam Sharma, Atharva Kulkarni, Tharun Suresh, Himanshi Mathur, Preslav Nakov, Md. Shad Akhtar, Tanmoy Chakraborty
| Challenge: | A common problem associated with meme comprehension lies in detecting the entities referenced and characterizing the role of each of these entities. |
| Approach: | They propose to use a memes dataset on US Politics and Covid-19 memes to characterize the role of harmful entities in memes. |
| Outcome: | The proposed model improves 4% over baseline and 1% over competing models. |
Towards Low-Resource Harmful Meme Detection with LMM Agents (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for harmful meme detection are limited due to the dynamic nature of memes . eliciting knowledge-revising behavior within the LMM agent is a key factor in achieving this goal . |
| Approach: | They propose an agency-driven framework for low-resource harmful meme detection . they use annotated memes to leverage label information as auxiliary signals for model . |
| Outcome: | The proposed framework achieves superior performance than state-of-the-art methods on the low-resource harmful meme detection task. |
MIND: A Multi-agent Framework for Zero-shot Harmful Meme Detection (2025.acl-long)
Copied to clipboard
| Challenge: | a rapid expansion of memes on social media highlights the need for effective methods to detect harmful content. |
| Approach: | They propose a multi-agent framework for zero-shot harmful meme detection that does not rely on annotated data. |
| Outcome: | The proposed framework outperforms existing zero-shot approaches on three meme datasets. |
Meme-ingful Analysis: Enhanced Understanding of Cyberbullying in Memes Through Multimodal Explanations (2024.eacl-long)
Copied to clipboard
| Challenge: | Recent laws like “right to explanations” have spurred research in developing interpretable models . a recent study has shown that multimodal explanations improve performance in generating textual justifications . |
| Approach: | They propose to use visual and textual modalities to explain why a given meme is cyberbullying . they use a Contrastive Language-Image Pretraining approach to generate textual justifications . |
| Outcome: | The proposed model improves performance in visual and textual explanations and identifies the visual evidence supporting a decision. |
A Context-Aware Contrastive Learning Framework for Hateful Meme Detection and Segmentation (2025.findings-naacl)
Copied to clipboard
| Challenge: | Empirical experiments show HateSieve surpasses existing LMMs in performance with fewer trainable parameters . |
| Approach: | They propose a framework to enhance detection and segmentation of hateful elements in memes by creating a triplet dataset and an Image-Text Alignment module. |
| Outcome: | HateSieve features a new framework that creates semantically correlated memes and generates contextual embeddings for accurate meme segmentation. |