Papers by Taku Hasegawa
Scene-Text Aware Image and Text Retrieval with Dual-Encoder (2022.acl-srw)
Copied to clipboard
| Challenge: | Existing studies on image and text retrieval using a dual-encoder model have not shown their effectiveness for fast inferences. |
| Approach: | They propose a dual-encoder model that connects vision and language in the same semantic space and integrates scene-text and visual information into a model. |
| Outcome: | The proposed model can interpret scene-text and surrounding visual information better than cross-encoder models. |
DueT: Image-Text Contrastive Transfer Learning with Dual-adapter Tuning (2023.emnlp-main)
Copied to clipboard
| Challenge: | Comparative learning models for vision and language models are gaining popularity . dueT trains only adapters inserted into pre-trained image and text encoders . |
| Approach: | They propose a transfer learning method for vision and language models built by contrastive learning that trains only adapters inserted into the frozen image and text encoders. |
| Outcome: | The proposed method outperforms fine-tuning, and the LoRA-based adapter method in English and Japanese domains. |