Papers by Idan Schwartz
Describing Sets of Images with Textual-PCA (2022.findings-emnlp)
Copied to clipboard
| Challenge: | a new method to describe images using a common theme is needed to describe the images . a grammatical phrase is not sufficient to describe an image set, since captioning engines are not general enough. |
| Approach: | They propose a method to capture attributes of images and variations within a set . they use a pretrained vision-language model to generate a centroid phrase with the largest average similarity . |
| Outcome: | The proposed method captures the essence of image sets and describes them in a semantically meaningful way . it is easy for humans to identify and describe a common theme, but it is not generic enough . |
Ensemble of MRR and NDCG models for Visual Dialog (2021.naacl-main)
Copied to clipboard
| Challenge: | BLEU scores favor correct syntax over semantics. |
| Approach: | They propose a non-parametric ranking method that integrates the ranks of two strong MRR and NDCG models into a single ranking that excels on both metrics. |
| Outcome: | The proposed model can keep the MRR and NDCG models state-of-the-art and the NDGC models state of the art. |
LaMI: Augmenting Large Language Models via Late Multi-Image Fusion (2026.acl-short)
Copied to clipboard
| Challenge: | Large Language Models lack visual grounding on visual reasoning, despite training on text alone. |
| Approach: | They propose a late multi-image fusion method that augments LLMs with test-time visual signals. |
| Outcome: | Using a late multi-image fusion method, the proposed model outperforms LLMs on visual reasoning and matches VLMs in vision-based tasks. |