Uncertainty-Calibrated Elastic Alignment for Multimodal Sentiment Analysis with Missing Modalities (2026.findings-acl)
Copied to clipboard
Kang He, Yuzhe Ding, Rao Fu, Yukang Feng, Kaipeng Zhang, Yiming Liu, Fei Li, Chong Teng, Donghong Ji
| Challenge: | Existing methods for multimodal sentiment analysis are often dynamically incomplete. |
| Approach: | They propose a new uncertainty-calibrated elastic alignment framework to address these issues by employing probabilistic imputation to capture cross-modal ambiguity and leverage the estimated uncertainty to drive elastic alignment. |
| Outcome: | The proposed framework outperforms state-of-the-art models in multiple benchmarks and consistently outperformed existing models. |
Similar Papers
Mitigating Inconsistencies in Multimodal Sentiment Analysis under Uncertain Missing Modalities (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies ignore the inconsistency phenomenon of missing modality in multimodal sentiment analysis . neglect of missing modalities may lead to incorrect semantic results . |
| Approach: | They propose an ensemble-based Missing Modality Reconstruction network to detect and recover missing modality features. |
| Outcome: | The proposed method is superior to existing methods on CMU-MOSI and IEMOCAP datasets. |
Proxy-Driven Robust Multimodal Sentiment Analysis with Incomplete Data (2025.acl-long)
Copied to clipboard
| Challenge: | Existing studies focus on optimizing model structures to handle uncertain missingness, but models still face challenges when dealing with uncertain missing data. |
| Approach: | They propose a data-centric robust multimodal sentiment analysis method, Proxy-Driven Robust Multimodal Fusion, which maps unimodal data to the latent space of Gaussian distributions to capture core features and structure. |
| Outcome: | The proposed method outperforms existing models in noise resistance and achieves state-of-the-art performance on multiple benchmark datasets. |
DEAR: Distributional Error-Aware Reliability for Robust Multimodal Sentiment Analysis with Missing Modalities (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods focus on feature completion but neglect semantic shifts caused by distribution gaps and decision risks under high uncertainty. |
| Approach: | They propose a distributional error-aware reliability estimation framework for robust MSA . they propose reconstructed features to be explicitly aligned with original distributional manifold . |
| Outcome: | The proposed framework mitigates semantic shifts by aligning reconstructed features with original distributional manifold . Extensive experiments on MOSI, MOSEI, and SIMS validate the framework . |
Uncertainty-Aware Cross-Modal Alignment for Hate Speech Detection (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for detecting hate speech ignore misalignment and uncertainty between modalities . social media platforms have become conduits for the rapid dissemination of hate speech . |
| Approach: | They propose an uncertainty-aware cross-modal alignment framework for hate speech detection that minimizes the misalignment of image and text in memes. |
| Outcome: | The proposed framework produces a competitive performance compared with existing methods. |
Which is Making the Contribution: Modulating Unimodal and Cross-modal Dynamics for Multimodal Sentiment Analysis (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies focus on learning cross-modal dynamics, but neglect to explore optimal solution for unimodal networks. |
| Approach: | They propose a new MSA framework to identify contribution of modalities and reduce impact of noisy information. |
| Outcome: | The proposed model outperforms state-of-the-art methods on publicly available datasets. |
Analyzing Modality Robustness in Multimodal Sentiment Analysis (2022.naacl-main)
Copied to clipboard
| Challenge: | despite its importance, little attention has been paid to improving the robustness of multimodal models. |
| Approach: | They propose simple diagnostic checks for modality robustness in a trained multimodal model . they find MSA models highly sensitive to a single modality, which creates issues . |
| Outcome: | The proposed checks show that models are highly sensitive to a single modality, which creates issues in their robustness. |
Missing Modality meets Meta Sampling (M3S): An Efficient Universal Approach for Multimodal Sentiment Analysis with Missing Modality (2022.aacl-main)
Copied to clipboard
| Challenge: | Existing methods to address missing modalities often assume a particular modality is completely missing due to recording or transmission error. |
| Approach: | They propose a missing modality-based meta-sampling approach for multimodal sentiment analysis with missing modalities . they conduct experiments on IEMOCAP, SIMS and CMU-MOSI datasets . |
| Outcome: | The proposed method significantly improves on existing models with a mixture of missing modalities. |
Beyond Static Alignment: Adaptive Arbitration for Semantic Incongruence in Semi-Supervised Multimodal Sentiment Analysis (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for semantic incongruence in sentiment analysis are limited by label-limited settings. |
| Approach: | They propose a framework for semi-supervised multimodal sentiment analysis that emphasizes stable cross-modal representations and reliable supervision. |
| Outcome: | The proposed framework outperforms state-of-the-art methods under label-limited settings. |
InfoEnh: Towards Multimodal Sentiment Analysis via Information Bottleneck Filter and Optimal Transport Alignment (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for multi-modal sentiment analysis have been developed to overcome these challenges. |
| Approach: | They propose a method that utilizes a masking technique as the bottleneck for information filtering and integrates all modalities into a common feature space via domain adaptation. |
| Outcome: | Extensive experiments on two benchmark MSA datasets show the proposed method performs better than baselines. |
UniMSE: Towards Unified Multimodal Sentiment Analysis and Emotion Recognition (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies study sentiment and emotion separately and do not fully exploit the complementary knowledge behind the two. |
| Approach: | They propose a multimodal sentiment knowledge-sharing framework that unifies MSA and ERC tasks from features, labels, and models. |
| Outcome: | The proposed framework achieves consistent improvements on four public benchmark datasets on MOSI, MOSEI, MELD, and IEMOCAP. |