The Devil is in the Distributions: Explicit Modeling of Scene Content is Key in Zero-Shot Video Captioning (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods for zero-shot video captioning focus on one key aspect of the scene and ignore the rest of the visual input. |
| Approach: | They propose a novel textual prompting strategy for zero-shot video captioning that uses a category-aware retrieval mechanism to promote prompt diversity while ensuring visual relevance. |
| Outcome: | The proposed method outperforms existing methods on in-domain and cross-domain settings. |
Similar Papers
MultiCapCLIP: Auto-Encoding Prompts for Zero-Shot Multilingual Visual Captioning (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods for supervised visual captioning require large scale of images or videos paired with descriptions in a specific language. |
| Approach: | They propose a zero-shot approach that generates captions for different scenarios without labeling . they use concept prompts to retrieve concepts and auto-encode them to learn writing styles . |
| Outcome: | The proposed approach generates captions for different scenarios and languages without labeled vision-caption pairs. |
ELIOT: Zero-Shot Video-Text Retrieval through Relevance-Boosted Captioning and Structural Information Extraction (2025.naacl-srw)
Copied to clipboard
| Challenge: | Recent advances in video-text retrieval (VTR) have relied on supervised learning and fine-tuning. |
| Approach: | They propose a zero-shot video-text retrieval framework that leverages off-the-shelf captioners, large language models, and text retrieval methods without additional training or annotated data. |
| Outcome: | The proposed framework outperforms existing methods on video-text retrieval benchmarks without data. |
Detecting and Mitigating Challenges in Zero-Shot Video Summarization with Video LLMs (2025.findings-acl)
Copied to clipboard
Luca Cagliero, Lorenzo Vaiani, Eliana Pastor, Alkis Koudounas, Elena Baralis, Vittorio Mazzia, Sandro Pollastrini, Thomas Gueudre, Manuel Giollo, Daniele Amberti, Yue Wu
| Challenge: | Video Large Language Models (VLLMs) exhibit impressive zero-shot capabilities in video analysis, but their performance varies significantly depending on the LLM prompt, the characteristics of the video, and the properties of the training data and LLM architecture. |
| Approach: | They propose to use Chain-of-Thought prompting to inject knowledge extracted by external, lightweight models into video summarization benchmarks to evaluate their performance. |
| Outcome: | The proposed solutions improve summarization performance by injecting knowledge extracted by external, lightweight models. |
A Video Is Worth 4096 Tokens: Verbalize Videos To Understand Them In Zero Shot (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing annotated training datasets hinder development of supervised learning models for multimedia content . lack of annotating benchmarks hinders development of models with satisfactory performance . a recent study shows that large language models have zero-shot performance in multimedia understanding . |
| Approach: | They propose to verbalize long videos to generate their descriptions in natural language . they then perform video-understanding tasks on the generated story as opposed to the original video . |
| Outcome: | The proposed method achieves better results than baselines for video understanding. |
Unleashing the Power of Large Language Models in Zero-shot Relation Extraction via Self-Prompting (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for zero-shot Relation Extraction (RE) lack detailed, context-specific prompts for understanding various sentences and relations. |
| Approach: | They propose a framework that uses a three-stage diversity approach to prompt LLMs by generating multiple synthetic samples that encapsulate specific relations from scratch. |
| Outcome: | The proposed framework outperforms existing LLM-based zero-shot RE methods on benchmark datasets and shows that it produces high-quality synthetic data that enhances performance. |
TROPE: TRaining-Free Object-Part Enhancement for Seamlessly Improving Fine-Grained Zero-Shot Image Captioning (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to enhance zero-shot abilities in image captioning fail with fine-grained datasets. |
| Approach: | They propose a method to enhance captions with additional object-part details using object detector proposals and natural language processing techniques. |
| Outcome: | The proposed method improves performance on fine-grained datasets and improves on existing methods. |
Zero-shot Approach to Overcome Perturbation Sensitivity of Prompts (2023.acl-long)
Copied to clipboard
| Challenge: | Recent studies have demonstrated that natural-language prompts can help to leverage the knowledge learned by pre-trained language models for the binary sentence-level sentiment classification task. |
| Approach: | They propose to use few-shot learning settings to fine-tune the sentiment classification model using manual or automatically generated prompts. |
| Outcome: | The proposed method outperforms the base prompt and the prompts generated using few-shot learning for the binary sentence-level sentiment classification task. |
ZeLa: Advancing Zero-Shot Multilingual Semantic Parsing with Large Language Models and Chain-of-Thought Strategies (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing approaches to augment multilingual datasets with labeled English data are lacking in annotated data. |
| Approach: | They propose a framework to augment English data and then use it to train parsers . they propose to use multilingual chain-of-thought prompting techniques to augment other languages' data . |
| Outcome: | The proposed framework augments English data in other languages and trains them with no demonstration samples in target languages. |
LMCap: Few-shot Multilingual Image Captioning by Retrieval Augmented Language Model Prompting (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing captioning models are mostly available on English or are specialised for other high-resource languages. |
| Approach: | They propose an image-blind few-shot multilingual captioning model that prompts a language model with retrieved captions. |
| Outcome: | The proposed model is competitive with fully-supervised multilingual captioning models without requiring any supervised training on captioning data. |
Large Language Models can Share Images, Too! (2024.findings-acl)
Copied to clipboard
| Challenge: | Using a zero-shot prompting, large language models can be used to share images in a multi-tasking environment. |
| Approach: | They introduce a dataset that includes enriched annotations and a framework to evaluate LLMs. |
| Outcome: | The proposed framework unlocks image-sharing capability of LLMs in zero-shot prompting, with ChatGPT achieving the best performance. |