Papers by Ajinkya Kale
Advancing Vision-Language Models with Adapter Ensemble Strategies (2024.findings-emnlp)
Copied to clipboard
| Challenge: | CLIP revolutes vision-language pretraining by using contrastive learning on paired web data. |
| Approach: | They propose to combine a "adapter ensemble" with traditional machine learning techniques to augment large-scale pretrained vision-language models. |
| Outcome: | The proposed model outperforms baselines and derives improvement when the number of ensemble parameters increases. |
Fine-grained Image Captioning with CLIP Reward (2022.findings-naacl)
Copied to clipboard
| Challenge: | Modern image captioning models are usually trained with text similarity objectives . reference captions often describe only the most salient objects in images . |
| Approach: | They propose to use CLIP to calculate multi-modal similarity and use it as a reward function . they propose a simple finetuning strategy to improve grammar that does not require extra text annotation. |
| Outcome: | The proposed model generates more distinctive captions than the CIDEroptimized model on text-to-image retrieval and fineCapEval. |
Search Query Language Identification Using Weak Labeling (2020.lrec-1)
Copied to clipboard
| Challenge: | a recent study has shown that language identification is a well-known task for natural language documents. |
| Approach: | They propose a search query language identification task that trains large-scale query-language pairs for training without loss of generalization. |
| Outcome: | The proposed model outperforms open domain model baselines by a large margin. |