Abstractive Multi-Video Captioning: Benchmark Dataset Construction and Extensive Evaluation (2024.lrec-main)
Copied to clipboard
| Challenge: | Abstractive multi-video captioning focuses on abstracting multiple videos with natural language. |
| Approach: | They propose a task that generates an abstract caption of shared video content . they propose end-to-end and cascade approaches to abstractive multi-video captioning . |
| Outcome: | The proposed task generates an abstract caption of shared content in a video group containing multiple videos. |
Similar Papers
A Challenging Multimodal Video Summary: Simultaneously Extracting and Generating Keyframe-Caption Pairs from Video (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to summarize video content have only considered video and image data, and the trend towards multimodal video summarization is changing. |
| Approach: | They propose a multimodal video summarization task setting and a dataset to train and evaluate the task. |
| Outcome: | The proposed task is useful as a practical application and presents a highly challenging problem worthy of study. |
Multimodal Abstractive Summarization for How2 Videos (P19-1)
Copied to clipboard
| Challenge: | Abstractive summarization is a task of producing a shorter version of the content in the document while preserving its information. |
| Approach: | They propose a new evaluation metric that measures semantic adequacy rather than fluency of abstractive summarization tasks. |
| Outcome: | The proposed model integrates information from different sources into a coherent output. |
Evaluation of Multilingual Image Captioning: How far can we get with CLIP models? (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing approaches to evaluate image captions are English-centric, despite improvements in the CLIPScore metric . however, there are no available benchmarks for multilingual captioning evaluation . |
| Approach: | They propose to use machine-translated and machine-repurposed datasets to evaluate CLIPScore variants in multilingual settings. |
| Outcome: | The proposed evaluation strategies are based on machine-translated and human judgements. |
End-to-end Dense Video Captioning as Sequence Generation (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods for dense video captioning use a two-stage generative process . but, more complex tasks are not able to fully utilize this powerful paradigm . |
| Approach: | They propose to model two subtasks of dense video captioning as one sequence generation task and predict the events and the corresponding descriptions. |
| Outcome: | Experiments on YouCook2 and ViTT show that the proposed model can be used on any video platform. |
Putting Captions to the Test: Evaluating Video Caption Quality through Multiple-Choice Question Answering (2026.acl-long)
Copied to clipboard
| Challenge: | Existing metrics for video captioning are based on text-based comparisons with ground-truth references. |
| Approach: | They propose a reference-free benchmark that assesses video captions based on their utility . they will release the benchmark to facilitate reproducible research . |
| Outcome: | The proposed benchmark improves on human-verified, fine-grained questions . it correlates significantly better with human judgments than existing metrics . |
Video2Commonsense: Generating Commonsense Descriptions to Enrich Video Captioning (2020.emnlp-main)
Copied to clipboard
| Challenge: | Observable changes in the scene are reflected in captions, but actions are also linked to social aspects such as intentions, effects, and attributes that describe the agent. |
| Approach: | They propose to generate captions from videos that describe latent aspects of the human agent's actions. |
| Outcome: | The proposed model can be used to describe latent aspects of human actions in video clips and answer questions about videos. |
Evaluating Multimodal Large Language Models on Video Captioning via Monte Carlo Tree Search (2025.acl-long)
Copied to clipboard
Linhao Yu, Xingguang Ji, Yahui Liu, Fanheng Kong, Chenxi Sun, Jingyuan Zhang, Hongzhi Zhang, V. W., Fuzheng Zhang, Deyi Xiong
| Challenge: | Existing benchmarks and evaluation protocols suffer from inadequate or homogeneous creation of key points, exorbitant cost of data creation, and limited evaluation scopes. |
| Approach: | They propose an automatic framework which leverages Monte Carlo Tree Search to construct numerous and diverse descriptive sentences that thoroughly represent video content in an iterative way. |
| Outcome: | The proposed framework improves MCTS-VCB and DREAM-1K on video captioning tasks by 25.0% and 16.3% respectively. |
VIEWS: Entity-Aware News Video Captioning (2024.emnlp-main)
Copied to clipboard
Hammad Ayyubi, Tianqi Liu, Arsha Nagrani, Xudong Lin, Mingda Zhang, Anurag Arnab, Feng Han, Yukun Zhu, Xuande Feng, Kevin Zhang, Jialu Liu, Shih-Fu Chang
| Challenge: | Existing video captioning benchmarks and models produce generic captions for videos that lack specific identification of individuals, locations, or organizations. |
| Approach: | They propose a task of directly summarizing news videos into captions that are entity-aware . they validate the effectiveness of their approach across three video captioning models . |
| Outcome: | The proposed approach is effective across three video captioning models. |
Learning-based Composite Metrics for Improved Caption Evaluation (P18-3)
Copied to clipboard
| Challenge: | Existing image captioning metrics focus on linguistic aspects and do not match human judgements at sentence-level. |
| Approach: | They propose to incorporate lexical and semantic metrics as features to capture adequacy and fluency of captions at different linguistic levels. |
| Outcome: | The proposed framework captures adequacy and fluency of captions at different linguistic levels. |
Abstractive Unsupervised Multi-Document Summarization using Paraphrastic Sentence Fusion (C18-1)
Copied to clipboard
| Challenge: | a new method for abstractive summarization is being developed for document summarizing . abstractive methods require extensive natural language generation to rewrite the sentences . |
| Approach: | They propose an unsupervised abstractive summarization system in multi-document context . they use a paraphrastic sentence fusion model which performs sentence synthesis and paraphrazing . |
| Outcome: | The proposed model improves information coverage and abstractiveness of generated sentences. |