Papers by Lihua He
Knowledge-Guided Dynamic Modality Attention Fusion Framework for Multimodal Sentiment Analysis (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods focus on equally treating the contribution of each modality or statically using text as the dominant modality to conduct interaction, which neglects the situation where each modal may become dominant. |
| Approach: | They propose a Knowledge-Guided Dynamic Modality Attention Fusion Framework (KuDA) that uses sentiment knowledge to guide the model dynamically selecting the dominant modality and adjusting the contributions of each modality. |
| Outcome: | The proposed model can be used to highlight the contribution of dominant modality through the correlation evaluation loss. |