| Challenge: | Existing methods for automatic caption generation of images are lacking in the field of image-related applications. |
| Approach: | They propose a method for automatically generating captions for news images . they propose several deep neural network architectures built upon Recurrent Neural Networks . |
| Outcome: | The proposed method outperforms a traditional method on a BBC News dataset using automatic evaluation and human evaluation. |
Similar Papers
Image Caption Generation for News Articles (2020.coling-main)
Copied to clipboard
| Challenge: | Existing work on news-image captioning requires a joint understanding of image and text. |
| Approach: | They propose a Transformer model that integrates text and image modalities and attends to textual features from visual features in generating a caption. |
| Outcome: | The proposed model outperforms the state-of-the-art model and improves the quality of news-image captions. |
Generating Image Captions in Arabic using Root-Word Based Recurrent Neural Networks and Deep Neural Networks (N18-4)
Copied to clipboard
| Challenge: | Existing studies on image caption generation in English focus on Western languages, ignoring Semitic and Middle-Eastern languages like Arabic, Hebrew, Urdu and Persian. |
| Approach: | They propose to leverage the critical dependency of Arabic to generate Arabic captions using root-word based Recurrent Neural Network and Deep Neural networks. |
| Outcome: | The proposed model outperforms English-Arabic translated captions on a dataset from newspapers in the Middle East. |
Entity-aware Image Caption Generation (D18-1)
Copied to clipboard
| Challenge: | Existing image captioning approaches generate generic descriptions of visual content and ignore background information. |
| Approach: | They propose a task which generates informative image captions using images and hashtags as input. |
| Outcome: | The proposed model outperforms unimodal baselines significantly with evaluation metrics on a dataset from Flickr. |
Exploring the Impact of Vision Features in News Image Captioning (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent state-of-art models can achieve competitive performance even without vision features. |
| Approach: | They conduct extensive experiments with mainstream news image captioning models to determine whether vision features contribute to the generation of captions. |
| Outcome: | The proposed models can achieve competitive performance even without vision features. |
Focus! Relevant and Sufficient Context Selection for News Image Captioning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work only coarsely leverages the article to extract the necessary context, which makes it difficult for models to identify relevant events and named entities. |
| Approach: | They propose to use a vision and language retrieval model CLIP to localize the visually grounded entities in the news article and then capture the non-visual entities via an open relation extraction model. |
| Outcome: | The proposed model significantly improves on existing models and achieves state-of-the-art on multiple benchmarks. |
Visual News: Benchmark and Challenges in News Image Captioning (2021.emnlp-main)
Copied to clipboard
| Challenge: | Visual News Captioner is an entity-aware model for news image captioning . Unlike standard image captions, news images depict situations where people, locations, and events are of paramount importance. |
| Approach: | They propose a visual news captioner model that integrates visual and textual features to generate captions with richer information such as events and entities. |
| Outcome: | The proposed model can generate captions with richer information such as events and entities. |
Bridging by Word: Image Grounded Vocabulary Construction for Visual Captioning (P19-1)
Copied to clipboard
| Challenge: | Existing research on image captioning generates frequent n-grams with irrelevant words. |
| Approach: | They propose to construct an image-grounded vocabulary incorporating visual information and relations among words into the decoding process directly. |
| Outcome: | The proposed framework is compared with state-of-the-art models on MS COCO and Flickr30k and shows that it is more efficient than existing models. |
Improving Image Captioning with Better Use of Caption (2020.acl-main)
Copied to clipboard
| Challenge: | Existing approaches to image captioning focus on visual attention, but many do not. |
| Approach: | They propose a framework that explores semantics available in captions and leverages that to enhance both image representation and caption generation. |
| Outcome: | The proposed framework outperforms baselines on the MSCOCO dataset and is state-of-the-art under a wide range of evaluation metrics. |
A Case Study on Neural Headline Generation for Editing Support (N19-2)
Copied to clipboard
Kazuma Murao, Ken Kobayashi, Hayato Kobayashi, Taichi Yatsuka, Takeshi Masuyama, Tatsuru Higurashi, Yoshimune Tabuchi
| Challenge: | a news-aggregator is a website or mobile application that aggregates web content . dozens of professional editors manually create their headlines, which are much shorter than the original headlines. |
| Approach: | They propose a neural headline generation model that automatically generates short headlines from news articles. |
| Outcome: | The proposed model is deployed to an editing support tool and compares editors' behavior before and after the release. |
Visually-Aware Context Modeling for News Image Captioning (2024.naacl-long)
Copied to clipboard
| Challenge: | a new framework for News Image Captioning emphasizes the connection between textual context and visual elements. |
| Approach: | They propose a face-naming module for learning better name embeddings from news images . they use CLIP to retrieve sentences that are semantically close to the image . |
| Outcome: | The proposed framework outperforms the current state-of-the-art by 7.97/5.80 CIDEr scores on GoodNews/NYTimes800k. |