MetaSumPerceiver: Multimodal Multi-Document Evidence Summarization for Fact-Checking (2024.acl-long)
Copied to clipboard
| Challenge: | Fact-checking claims on social media platforms poses a significant challenge due to the large volume of new claims constantly being posted without sufficient methods for verification. |
| Approach: | They propose a model that generates claim-specific summaries from multimodal multi-document datasets using a perceiver-based model that is able to handle inputs from multiple modalities of arbitrary lengths. |
| Outcome: | The proposed model outperforms the SOTA approach by 4.6% in the claim verification task on the MOCHEG dataset and shows strong performance on the new multi-document claims dataset. |
Similar Papers
Harnessing Abstractive Summarization for Fact-Checked Claim Detection (2022.coling-1)
Copied to clipboard
| Challenge: | Social media platforms are becoming battlegrounds for anti-social elements . fact-checking organizations cannot cope with the rapid dissemination of misinformation . a new workflow for fact- checking can be implemented to reduce human time for tasks with high cognition . |
| Approach: | They propose a workflow for detecting previously fact-checked claims that uses abstractive summarization to generate crisp queries. |
| Outcome: | The proposed workflow achieves Recall@5 and MRR of 35% and 0.3, respectively. |
Multimodal Fact-Checking with Vision Language Models: A Probing Classifier based Solution with Embedding Strategies (2025.coling-main)
Copied to clipboard
| Challenge: | Existing fact-checking systems that use text and image information are susceptible to fake news spread by social media platforms. |
| Approach: | They propose a neural probing classifier based on multimodality and embeddings from text and image encoders to represent multimodal content for fact-checking. |
| Outcome: | The proposed classifier outperforms KNN and SVM baselines in leveraging extracted embeddings, highlighting its effectiveness for multimodal fact-checking. |
Multi-doc Hybrid Summarization via Salient Representation Learning (2023.acl-industry)
Copied to clipboard
| Challenge: | Multi-document summarization is gaining more and more attention . extractive multi-doc approaches intend to directly extract key facts from multiple sources . |
| Approach: | They propose a multi-document hybrid summarization approach that generates a human-readable summary and extracts corresponding key evidences based on multi-doc inputs. |
| Outcome: | The proposed method generates a human-readable summary and extracts key evidences based on multi-doc inputs. |
FACTIFY3M: A benchmark for multimodal fact verification with explainability through 5W Question-Answering (2023.emnlp-main)
Copied to clipboard
Megha Chakraborty, Khushbu Pahwa, Anku Rani, Shreyas Chatterjee, Dwip Dalal, Harshit Dave, Ritvik G, Preethi Gurumurthy, Adarsh Mahor, Samahriti Mukherjee, Aditya Pakala, Ishan Paul, Janvita Reddy, Arghya Sarkar, Kinjal Sensharma, Aman Chadha, Amit Sheth, Amitava Das
| Challenge: | Disinformation can cause disruption in the share market, panic and anxiety in society, and even death during crises. |
| Approach: | a new dataset is being developed to help combat disinformation . the dataset is a multimodal fake news dataset with 5W question-answering . |
| Outcome: | FACTIFY 3M is the largest dataset and benchmark for multimodal fact verification. |
M3D: MultiModal MultiDocument Fine-Grained Inconsistency Detection (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for fact-checking claim are limited by ambiguous information and lack sample-level predictions. |
| Approach: | They propose a method that predicts the logical relationship of each aspect of a claim from a set of multimodal documents. |
| Outcome: | The proposed method outperforms existing models on two benchmarks while providing finer-grained predictions, explanations, and evidence. |
Multimodal Automated Fact-Checking: A Survey (2023.findings-emnlp)
Copied to clipboard
Mubashara Akhtar, Michael Schlichtkrull, Zhijiang Guo, Oana Cocarascu, Elena Simperl, Andreas Vlachos
| Challenge: | Existing studies on automated fact-checking focus on text, but they focus on a single modality, text . multimodal misinformation is perceived as more credible by humans and spreads faster than text-only counterparts. |
| Approach: | They propose a framework for automated fact-checking that includes subtasks unique to multimodal misinformation. |
| Outcome: | The proposed framework includes subtasks unique to multimodal misinformation. |
HintsOfTruth: A Multimodal Checkworthiness Detection Dataset with Real and Synthetic Claims (2025.acl-long)
Copied to clipboard
| Challenge: | Identifying checkworthy claims is the first step, but detection methods struggle with content that is (1) multimodal, (2) from diverse domains, and (3) synthetic. |
| Approach: | They propose a dataset for multimodal checkworthiness detection with 27K real-world and synthetic image/claim pairs. |
| Outcome: | The proposed dataset compares lightweight text-based encoders to multimodal models but only focus on claim-like content. |
ACQUIRED: A Dataset for Answering Counterfactual Questions In Real-Life Videos (2023.emnlp-main)
Copied to clipboard
Te-Lin Wu, Zi-Yi Dou, Qingyuan Hu, Yu Hou, Nischal Chandra, Marjorie Freedman, Ralph Weischedel, Nanyun Peng
| Challenge: | despite its importance, there are few datasets that cover multimodal counterfactual reasoning . a dataset focusing on this area is limited because of its limited coverage over synthetic environments . |
| Approach: | They develop a video question answering dataset that provides questions on multimodal reasoning . they ask questions about counterfactual hypotheses over visual events . |
| Outcome: | The proposed dataset shows a significant performance gap between models and humans . it provides questions that span physical, social, and temporal dimensions . |
Benchmarking the Generation of Fact Checking Explanations (2023.tacl-1)
Copied to clipboard
| Challenge: | Automating fact-checking is a time-consuming task that cannot keep up with the ever-increasing amount of fake news produced daily. |
| Approach: | They propose to automate the process of fact-checking by generating justifications from textual explanations of why a claim is classified as either true or false. |
| Outcome: | The proposed approach improves summarization performance over unstructured knowledge and with two datasets with different styles and structures. |
MultiFC: A Real-World Multi-Domain Dataset for Evidence-Based Fact Checking of Claims (D19-1)
Copied to clipboard
Isabelle Augenstein, Christina Lioma, Dongsheng Wang, Lucas Chaves Lima, Casper Hansen, Christian Hansen, Jakob Grue Simonsen
| Challenge: | Existing efforts to verify factual claims are limited by small datasets or artificially constructed datasets. |
| Approach: | They propose to use the largest publicly available dataset of naturally occurring factual claims for automatic claim verification. |
| Outcome: | The proposed model outperforms baseline models and evidence pages significantly. |