Debiasing Multimodal Models via Causal Information Minimization (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for debiasing multimodal models use approximate heuristics to represent the biases, such as shallow features from early stages of training or unimodal features for multimodal tasks like VQA, which may not be accurate. |
| Approach: | They propose a method that leverages causally-motivated information minimization to learn the confounder representations of a causal graph for multimodal data. |
| Outcome: | The proposed method improves out-of-distribution performance on multiple multimodal datasets without sacrificing in-distance performance. |
Similar Papers
Quantifying and Mitigating Unimodal Biases in Multimodal Large Language Models: A Causal Perspective (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in Large Language Models have facilitated the development of Multimodal LLMs. |
| Approach: | They propose a causal framework to interpret unimodal biases in visual question answering problems and a framework to integrate information from different modalities and mitigate biase. |
| Outcome: | The proposed framework analyzes visual question answering (VQA) problems to assess their impact on predictions. |
Multimodal Clickbait Detection by De-confounding Biases Using Causal Representation Inference (2024.emnlp-main)
Copied to clipboard
| Challenge: | a new method to detect clickbait posts on the Web is needed to detect such posts. |
| Approach: | They propose a method to detect clickbait posts on the Web using latent factors . they use features in multiple modalities to characterize the posts and causal inference to eliminate noise . |
| Outcome: | The proposed method can detect clickbait posts on popular social media platforms with good generalization ability. |
Text and Causal Inference: A Review of Using Text to Remove Confounding from Causal Estimates (2020.acl-main)
Copied to clipboard
| Challenge: | Unmeasured or latent confounders can bias causal estimates and this has motivated interest in measuring potential confounder from observed text. |
| Approach: | They propose to use text to measure potential confounders in a way that allows for a rich measurement of multiple confounder variables. |
| Outcome: | The proposed method is based on an individual’s entire history of social media posts or the content of a news article. |
Counterfactual Augmentation for Multimodal Learning Under Presentation Bias (2023.findings-emnlp)
Copied to clipboard
| Challenge: | In real-world machine learning systems, labels are often derived from user behaviors that the system wishes to encourage. |
| Approach: | They propose a method for correcting presentation bias using generated counterfactual labels by augmentation of the labels by the user. |
| Outcome: | The proposed method improves performance in an oracle setting compared to uncorrected models and existing bias-correction methods. |
Beyond Spurious Signals: Debiasing Multimodal Large Language Models via Counterfactual Inference and Adaptive Expert Routing (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Multimodal Large Language Models (MLLMs) often rely on spurious correlations, undermining their robustness and generalization. |
| Approach: | They propose a causal mediation-based debiasing framework to address correlation bias in MLLMs . they distinguish core semantics from spurious textual and visual contexts using counterfactual examples . |
| Outcome: | The proposed framework surpasses existing state-of-the-art models on sarcasm detection and sentiment analysis tasks. |
Causal-Debias: Unifying Debiasing in Pretrained Language Models and Fine-tuning via Causal Invariant Learning (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods to remove unwanted stereotypical associations from pretrained language models (PLMs) are often focused on removing unwanted stereotypes from PLMs. |
| Approach: | They propose a framework to remove unwanted stereotypical associations in pretrained language models . they propose bias-relevant factors are causal, while labelrelevant factors causal . |
| Outcome: | The proposed framework reduces stereotypical associations after PLMs are fine-tuned . the proposed framework mitigates bias from a causal invariant perspective . |
DINER: Debiasing Aspect-based Sentiment Analysis with Multi-variable Causal Inference (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to debiase ABSA focus on single-variable causal inference . aspect-based sentiment analysis models are prone to learn spurious correlations from annotation biases . |
| Approach: | They propose a framework based on multivariable causal inference for debiasing ABSA . they propose to model different types of biases based upon different causal intervention methods . |
| Outcome: | The proposed framework tackles different types of biases based on different intervention methods. |
CoVariance-based Causal Debiasing for Entity and Relation Extraction (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Named Entity Recognition and Relation Extraction are key tasks of Information Extraction. |
| Approach: | They propose a causal framework called c ovariance and variance optimization framework (OVO) to optimize feature representations and conduct general debiasing. |
| Outcome: | The proposed framework minimizes characterizing features’ covariance for alleviating selection and distribution bias and enhances feature representation in the feature space. |
Learning Invariant Modality Representation for Robust Multimodal Learning from a Causal Inference Perspective (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to multimodal affective computing learn spurious correlations from training data rather than genuine causal relationships, harming generalization under distribution shifts or noisy modalities. |
| Approach: | They propose a causal modality-invariant representation framework that separates each modality into ‘causal invariant’ and ‘environment-specific spurious representation’ from a modal inference perspective. |
| Outcome: | Experiments on multiple multimodal benchmarks show that the proposed framework achieves state-of-the-art performance. |
A Robust Dual-debiasing VQA Model based on Counterfactual Causal Effect (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing VQA models suffer from language bias that indicates a spurious correlation between textual questions and answers. |
| Approach: | They propose a model agnostic dual-debiasing framework that models two types of language bias by separate branches under counterfactual inference framework. |
| Outcome: | The proposed framework significantly reduces language bias and achieves state-of-the-art performance on the benchmark datasets. |