Papers by Takumi Takada
Direct Metric Optimization for Image Captioning through Reward-Weighted Augmented Data Utilization (2024.acl-long)
Copied to clipboard
Takumi Takada, Yuma Suzuki, Hiroki Takushima, Hayato Tanoue, Haruki Sato, Aiswariya Kumar, Hiroki Nishihara, Takayuki Hori, Kazuya Ueki
| Challenge: | Recent large-scale vision language models (VLMs) lack continuity between learning objective and performance metrics. |
| Approach: | They propose a lightweight final-metric-optimizing training method that replaces the expensive exploration process in RL with an offline, diverse text data augmentation method. |
| Outcome: | The proposed method achieves comparable performance to state-of-the-art RL method while saving hundreds of times more model forwarding iterations and greater amounts of computation time. |