Papers by Shitian Zhao
Unleashing the Potentials of Likelihood Composition for Multi-modal Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing multi-modal language models with different architectures, parameter sizes, training datasets, and pipelines exhibit varying strengths across different tasks. |
| Approach: | They propose a framework for fusing heterogeneous models off-the-shell, which they call likelihood composition, and introduce basic operations to compose multiple models’ likelihood distribution when doing a multi-choice visual-question-answering task. |
| Outcome: | The proposed framework can be used to fusing heterogeneous models off-the-shell. |