COSMMIC: Comment-Sensitive Multimodal Multilingual Indian Corpus for Summarization and Headline Generation (2025.acl-long)
Copied to clipboard
Raghvendra Kumar, Mohammed Salman S A, Aryan Sahu, Tridib Nandi, Pragathi Y P, Sriparna Saha, Jose G Moreno
| Challenge: | COSMMIC is a multimodal, multilingual dataset featuring nine major Indian languages. |
| Approach: | They propose a multimodal, multilingual multimodal multimodal dataset that integrates text, images and user feedback to enhance summarization. |
| Outcome: | The proposed dataset is based on 4,959 article-image pairs and 24,484 reader comments with ground-truth summaries available in all included languages. |
Similar Papers
PMIndiaSum: Multilingual and Cross-lingual Headline Summarization for Languages in India (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing datasets for Indian languages are limited in terms of coverage and size. |
| Approach: | They propose a multilingual and massively parallel summarization corpus focused on languages in India that provides a training and testing ground for four language families, 14 languages, and the largest to date with 196 language pairs. |
| Outcome: | The proposed dataset provides a training and testing ground for four language families, 14 languages, and the largest to date with 196 language pairs. |
mRedditSum: A Multimodal Abstractive Summarization Dataset of Reddit Threads with Images (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing summarization datasets do not cover multimodal discussions, multiple modalities, or both . mRedditSum consists of 3,033 discussion threads and images with human-written summaries. |
| Approach: | They propose a multimodal discussion summarization dataset that annotates 3,033 discussion threads with a human-written summary. |
| Outcome: | The proposed method outperforms existing models and serves as competitive baseline for future work. |
Multimodality for NLP-Centered Applications: Resources, Advances and Frontiers (2022.lrec-1)
Copied to clipboard
| Challenge: | resurgence of multimodal datasets has attracted significant research interest, but there is no comprehensive survey for this task. |
| Approach: | They present a survey of a multimodal dataset with different modalities according to the applications. |
| Outcome: | The proposed datasets are available online and discuss the new frontier and motivate future researches. |
MSMO: Multimodal Summarization with Multimodal Output (D18-1)
Copied to clipboard
| Challenge: | Existing studies show that multimodal summarization can improve user satisfaction for informativeness of summaries by using information in visual modality. |
| Approach: | They propose a task to generate text and select the most relevant image from the multimodal input and a novel multimodal automatic evaluation method to evaluate multimodal outputs. |
| Outcome: | The proposed method improves user satisfaction by 12.4% compared to the current system . |
MULSUM: A Multimodal Summarization System with Vis-Aligner and Diversity-Aware Image Selection (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing systems that condense text and images into concise, faithful digests are inefficient and require large fusion transformers. |
| Approach: | They propose a framework that uses image embeddings to generate a visually informed text summary and a Diversity-Aware Image Selector to maximize images-relevance to the summary. |
| Outcome: | The proposed framework outperforms baselines on automatic metrics such as ROUGE and human evaluation shows that selected images act as explanatory evidence rather than ornamental add-ons. |
UrduMASD: A Multimodal Abstractive Summarization Dataset for Urdu (2024.lrec-main)
Copied to clipboard
| Challenge: | a surge of multimodal content on social media has transformed our methods of communication and information exchange. |
| Approach: | They propose a video-based Urdu multimodal abstractive text summarization dataset . it uses a variety of evaluation metrics to ensure the quality of the dataset amounted to a high quality one . |
| Outcome: | The proposed dataset surpasses existing datasets on key quality metrics. |
Multilingual Generation in Abstractive Summarization: A Comparative Study (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing models for multilingual generation lack thorough analysis due to extensive linguistic diversity. |
| Approach: | They propose to classify multilingual generation methodologies into three categories based on their underlying modeling principles . they introduce an automatic metric to mitigate spurious correlations associated with language mixing . |
| Outcome: | The proposed model improves in high-resource, low-resourced, and zero-shot scenarios. |
REFINESUMM: Self-Refining MLLM for Generating a Multimodal Summarization Dataset (2024.acl-long)
Copied to clipboard
| Challenge: | generating accurate and faithful multimodal summaries is challenging due to lack of appropriate multimodal datasets . large language models excel at synthesizing key information from diverse sources, but lack of adequate multimodal data sets for fine-tuning . |
| Approach: | They propose a dataset specifically designed for image-text multimodal summarization . they generate summaries from Wikipedia sections and corresponding images and evaluate them . |
| Outcome: | The proposed dataset improves summary quality by training a critic model on human annotations and using its predictions to remove low-quality summaries. |
Auto-hMDS: Automatic Construction of a Large Heterogeneous Multilingual Multi-Document Summarization Corpus (L18-1)
Copied to clipboard
| Challenge: | Existing datasets for automatic text summarization are small and focused on newswires. |
| Approach: | They propose to automatically generate a large multilingual multi-document summarization corpus using Wikipedia articles as summaries and to automatically search for appropriate source documents. |
| Outcome: | The proposed corpus contains 7,316 topics in English and German with different summary lengths and number of source documents. |
Enhancing Large Language Models for Scientific Multimodal Summarization with Multimodal Output (2025.coling-industry)
Copied to clipboard
| Challenge: | Scientific publications are becoming more multimedia, containing both text and visual content. |
| Approach: | They propose a framework for Scientific Multimodal Summarization with Multimodal Output . it leverages the power of large language models and extends its capability to cross-modal understanding . |
| Outcome: | The proposed framework outperforms uni- and multi-modality methods on two new datasets . it leverages the power of large language models and extends its capability to cross-modal understanding . |