Papers by Zhenbang Wu
MedCLIP: Contrastive Learning from Unpaired Medical Images and Text (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing vision-text contrastive learning methods encounter many false negatives, i.e., images and reports from separate patients probably carry the same semantics but are wrongly treated as negatives. |
| Approach: | They propose to decouple medical image-text contrastive learning and replace it with semantic matching loss based on medical knowledge to eliminate false negatives in contrastive training. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on zero-shot prediction, supervised classification, and image-text retrieval with only 20K pre-training data. |