Papers by Michal Yarom
Generating Coherent Sequences of Visual Illustrations for Real-World Manual Tasks (2024.acl-long)
Copied to clipboard
João Bordalo, Vasco Ramos, Rodrigo Valério, Diogo Glória-Silva, Yonatan Bitton, Michal Yarom, Idan Szpektor, Joao Magalhaes
| Challenge: | Large Vision/Language Models (LVLMs) are less capable of generating accompanying image sequences. |
| Approach: | They propose a method that integrates a Latent Diffusion Model (LDM) with an LLM to generate captions to maintain semantic coherence of the sequence. |
| Outcome: | The proposed method is preferred by humans in 46.6% of the cases against 26.6% for the second best method. |
MaXM: Towards Multilingual Visual Question Answering (2023.findings-emnlp)
Copied to clipboard
Soravit Changpinyo, Linting Xue, Michal Yarom, Ashish Thapliyal, Idan Szpektor, Julien Amelot, Xi Chen, Radu Soricut
| Challenge: | Visual Question Answering (VQA) has been studied in the English language, but in other languages it would require a considerable amount of resources. |
| Approach: | They propose scalable solutions to multilingual visual question answering using an English language framework and an annotation protocol. |
| Outcome: | The proposed framework reduces human annotation efforts and creates a test-only VQA benchmark in 7 languages. |