Firoj Alam, Stefano Cresci, Tanmoy Chakraborty, Fabrizio Silvestri, Dimiter Dimitrov, Giovanni Da San Martino, Shaden Shaar, Hamed Firooz, Preslav Nakov
| Challenge: | Recent years have witnessed the proliferation of offensive content online such as fake news, propaganda, misinformation, and disinformation. |
| Approach: | They propose to tackle online multimodal offensive content using different modalities and combinations thereof. |
| Outcome: | The proposed approach combines factuality and harmfulness in a framework that can be used for multiple modalities and combinations of modality. |
Similar Papers
Multimodal Automated Fact-Checking: A Survey (2023.findings-emnlp)
Copied to clipboard
Mubashara Akhtar, Michael Schlichtkrull, Zhijiang Guo, Oana Cocarascu, Elena Simperl, Andreas Vlachos
| Challenge: | Existing studies on automated fact-checking focus on text, but they focus on a single modality, text . multimodal misinformation is perceived as more credible by humans and spreads faster than text-only counterparts. |
| Approach: | They propose a framework for automated fact-checking that includes subtasks unique to multimodal misinformation. |
| Outcome: | The proposed framework includes subtasks unique to multimodal misinformation. |
Multimodal Misinformation Detection by Learning from Synthetic Data with Multimodal LLMs (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Obtaining large-scale, high-quality real-world fact-checking datasets is costly . generalizability of detectors trained on synthetic data to real-life scenarios remains unclear . |
| Approach: | They propose to use synthetic data to learn from real-world data to detect multimodal misinformation . they propose to combine model-agnostic data selection methods with real-life data distributions . |
| Outcome: | The proposed method improves the performance of a small MLLM on real-world fact-checking datasets, surpassing GPT-4V. |
Detecting Harmful Memes and Their Targets (2021.findings-acl)
Copied to clipboard
Shraman Pramanick, Dimitar Dimitrov, Rituparna Mukherjee, Shivam Sharma, Md. Shad Akhtar, Preslav Nakov, Tanmoy Chakraborty
| Challenge: | a growing body of research on meme analysis has focused on detecting harmful memes and their social entities . a meme is a form of content that is often harmless and designed to look funny . but its multimodal nature and camouflaged semantics make its analysis challenging . |
| Approach: | They propose to use multimodal models to detect harmful memes and identify social entities that harmful meme targets. |
| Outcome: | The proposed model can detect harmful memes and the social entities they target . the proposed model lacks the appropriate contexts and is poorly validated . |
Causal Intervention and Counterfactual Reasoning for Multi-modal Fake News Detection (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods for multi-modal fake news detection neglect the fact that some label-specific features cannot generalize well to the testing set, thus suffering from the latent data bias. |
| Approach: | They propose a Causal intervention and Counterfactual reasoning based debiasing framework for multi-modal fake news detection that eliminates the image-only bias by deducting the direct effect of the image from the total effect on labels. |
| Outcome: | The proposed framework eliminates the psycholinguistic bias in the text and the bias of inferring news label based on only image features. |
FACTIFY3M: A benchmark for multimodal fact verification with explainability through 5W Question-Answering (2023.emnlp-main)
Copied to clipboard
Megha Chakraborty, Khushbu Pahwa, Anku Rani, Shreyas Chatterjee, Dwip Dalal, Harshit Dave, Ritvik G, Preethi Gurumurthy, Adarsh Mahor, Samahriti Mukherjee, Aditya Pakala, Ishan Paul, Janvita Reddy, Arghya Sarkar, Kinjal Sensharma, Aman Chadha, Amit Sheth, Amitava Das
| Challenge: | Disinformation can cause disruption in the share market, panic and anxiety in society, and even death during crises. |
| Approach: | a new dataset is being developed to help combat disinformation . the dataset is a multimodal fake news dataset with 5W question-answering . |
| Outcome: | FACTIFY 3M is the largest dataset and benchmark for multimodal fact verification. |
Multimodality for NLP-Centered Applications: Resources, Advances and Frontiers (2022.lrec-1)
Copied to clipboard
| Challenge: | resurgence of multimodal datasets has attracted significant research interest, but there is no comprehensive survey for this task. |
| Approach: | They present a survey of a multimodal dataset with different modalities according to the applications. |
| Outcome: | The proposed datasets are available online and discuss the new frontier and motivate future researches. |
Not all Fake News is Written: A Dataset and Analysis of Misleading Video Headlines (2023.emnlp-main)
Copied to clipboard
| Challenge: | Social media platforms are used by half of U.S. adults for everyday news consumption. |
| Approach: | They propose to analyze video headlines and whether annotators believe the headline is representative of the video’s contents. |
| Outcome: | The proposed dataset analyzes video headlines and explains why annotators view a video as misleading. |
MM-Claims: A Dataset for Multimodal Claim Detection in Social Media (2022.findings-naacl)
Copied to clipboard
Gullal Singh Cheema, Sherzod Hakimov, Abdul Sittar, Eric Müller-Budack, Christian Otto, Ralph Ewerth
| Challenge: | Using image and text, we investigate the role of image and texts in fake news detection . claim detection is a step in fighting misinformation and as a precursor to prioritize potentially false information for fact-checking. |
| Approach: | They propose a dataset that consists of tweets and corresponding images for claim detection . they evaluate strong unimodal and multimodal baselines and analyze drawbacks of current models . |
| Outcome: | The proposed dataset evaluates strong unimodal and multimodal baselines and examines drawbacks of existing models. |
The Battlefront of Combating Misinformation and Coping with Media Bias (2022.aacl-tutorials)
Copied to clipboard
| Challenge: | a growing number of misinformation and misinformation is affecting our daily lives . a tutorial aims to address the challenges of detecting fake news and media bias . |
| Approach: | They provide an overview of the frontier in fighting misinformation . they propose to develop a robust fake news detection system to combat misinformation. |
| Outcome: | This tutorial examines the frontiers of fake news detection and media bias detection . it focuses on how to fact-check information pieces and uncover bias and agenda of news sources . |
Narratives at Conflict: Computational Analysis of News Framing in Multilingual Disinformation Campaigns (2024.acl-srw)
Copied to clipboard
| Challenge: | Existing methods for multilingual framing differ from those used in English-speaking world . framers often use loaded vocabularies to create political images or favor a particular point of view . |
| Approach: | They use eight years of Russian-backed disinformation campaigns to examine framing . they find that disinformation campaign consistently favors specific framers . |
| Outcome: | The proposed method underperforms and shows high disagreements in Russian-language articles . the proposed method is based on eight years of Russian-backed disinformation campaigns . |