Papers by Si-Woo Kim
IFCap: Image-like Retrieval and Frequency-based Entity Filtering for Zero-shot Captioning (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing text-only training methods overlook the modality gap between using text data during training and employing images during inference. |
| Approach: | They propose a novel approach that aligns text features with visually relevant features to mitigate the modality gap between using text data during training and employing images during inference. |
| Outcome: | The proposed method outperforms the state-of-the-art methods in image captioning and video captioning by a significant margin compared to training with text data. |
Sali4Vid: Saliency-Aware Video Reweighting and Adaptive Caption Retrieval for Dense Video Captioning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent work proposes end-to-end models but suffer from limitations . prior work focused on generating captions from long video streams . |
| Approach: | They propose a saliency-aware framework that localizes events and generates captions for each event. |
| Outcome: | The proposed framework achieves state-of-the-art results on YouCook2 and ViTT. |