Papers by Haiyan Fu
Prefix-diffusion: A Lightweight Diffusion Model for Diverse Image Captioning (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing image captioning models require large trainable parameters to bridge visual and textual representations. |
| Approach: | They propose a lightweight image captioning network in combination with continuous diffusion that injects prefix image embeddings into denoising process of diffusion model. |
| Outcome: | The proposed method generates diverse captions with relatively less parameters while maintaining fluency and relevance compared with other models. |