Papers by Jered McInerney
That’s the Wrong Lung! Evaluating and Improving the Interpretability of Unsupervised Multimodal Encoders for Medical Data (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent multimodal models induce soft local alignments between image regions and sentences. |
| Approach: | They compare alignments from a state-of-the-art multimodal model for EHR with human annotations that link image regions to sentences. |
| Outcome: | The proposed models induce soft local alignments between image regions and sentences . the text has an often weak or unintuitive influence on attention, the authors found . |