| Challenge: | Experimental results show that JoGANIC outperforms state-of-the-art methods for image caption generation. |
| Approach: | They propose a method to generate descriptive and informative captions for news article images . they leverage the structure of captions to improve the generation quality and guide their representation . |
| Outcome: | The proposed method outperforms state-of-the-art methods on two large-scale datasets. |
Similar Papers
Image Caption Generation for News Articles (2020.coling-main)
Copied to clipboard
| Challenge: | Existing work on news-image captioning requires a joint understanding of image and text. |
| Approach: | They propose a Transformer model that integrates text and image modalities and attends to textual features from visual features in generating a caption. |
| Outcome: | The proposed model outperforms the state-of-the-art model and improves the quality of news-image captions. |
Visual News: Benchmark and Challenges in News Image Captioning (2021.emnlp-main)
Copied to clipboard
| Challenge: | Visual News Captioner is an entity-aware model for news image captioning . Unlike standard image captions, news images depict situations where people, locations, and events are of paramount importance. |
| Approach: | They propose a visual news captioner model that integrates visual and textual features to generate captions with richer information such as events and entities. |
| Outcome: | The proposed model can generate captions with richer information such as events and entities. |
Entity-aware Image Caption Generation (D18-1)
Copied to clipboard
| Challenge: | Existing image captioning approaches generate generic descriptions of visual content and ignore background information. |
| Approach: | They propose a task which generates informative image captions using images and hashtags as input. |
| Outcome: | The proposed model outperforms unimodal baselines significantly with evaluation metrics on a dataset from Flickr. |
Focus! Relevant and Sufficient Context Selection for News Image Captioning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work only coarsely leverages the article to extract the necessary context, which makes it difficult for models to identify relevant events and named entities. |
| Approach: | They propose to use a vision and language retrieval model CLIP to localize the visually grounded entities in the news article and then capture the non-visual entities via an open relation extraction model. |
| Outcome: | The proposed model significantly improves on existing models and achieves state-of-the-art on multiple benchmarks. |
VIEWS: Entity-Aware News Video Captioning (2024.emnlp-main)
Copied to clipboard
Hammad Ayyubi, Tianqi Liu, Arsha Nagrani, Xudong Lin, Mingda Zhang, Anurag Arnab, Feng Han, Yukun Zhu, Xuande Feng, Kevin Zhang, Jialu Liu, Shih-Fu Chang
| Challenge: | Existing video captioning benchmarks and models produce generic captions for videos that lack specific identification of individuals, locations, or organizations. |
| Approach: | They propose a task of directly summarizing news videos into captions that are entity-aware . they validate the effectiveness of their approach across three video captioning models . |
| Outcome: | The proposed approach is effective across three video captioning models. |
Visually-Aware Context Modeling for News Image Captioning (2024.naacl-long)
Copied to clipboard
| Challenge: | a new framework for News Image Captioning emphasizes the connection between textual context and visual elements. |
| Approach: | They propose a face-naming module for learning better name embeddings from news images . they use CLIP to retrieve sentences that are semantically close to the image . |
| Outcome: | The proposed framework outperforms the current state-of-the-art by 7.97/5.80 CIDEr scores on GoodNews/NYTimes800k. |
AGIC: Attention-Guided Image Captioning to Improve Caption Relevance (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods for image captioning generate generic captions that are limited in capturing nuanced visual details. |
| Approach: | They propose attention-guided image captioning which amplifies visual regions directly in the feature space to guide caption generation. |
| Outcome: | The proposed approach matches or surpasses state-of-the-art models while achieving faster inference. |
Exploring the Impact of Vision Features in News Image Captioning (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent state-of-art models can achieve competitive performance even without vision features. |
| Approach: | They conduct extensive experiments with mainstream news image captioning models to determine whether vision features contribute to the generation of captions. |
| Outcome: | The proposed models can achieve competitive performance even without vision features. |
Neural Caption Generation for News Images (L18-1)
Copied to clipboard
| Challenge: | Existing methods for automatic caption generation of images are lacking in the field of image-related applications. |
| Approach: | They propose a method for automatically generating captions for news images . they propose several deep neural network architectures built upon Recurrent Neural Networks . |
| Outcome: | The proposed method outperforms a traditional method on a BBC News dataset using automatic evaluation and human evaluation. |
Improving Image Captioning with Better Use of Caption (2020.acl-main)
Copied to clipboard
| Challenge: | Existing approaches to image captioning focus on visual attention, but many do not. |
| Approach: | They propose a framework that explores semantics available in captions and leverages that to enhance both image representation and caption generation. |
| Outcome: | The proposed framework outperforms baselines on the MSCOCO dataset and is state-of-the-art under a wide range of evaluation metrics. |