DICA: Dual-Indicator Guided Contrastive Alignment in Multimodal Large Language Models

Challenge:	Multimodal large language models may deviate from this pattern due to attention drift and underutilization of visual evidence.
Approach:	They propose a Dual-Indicator Guided Contrastive Alignment (DICA) that tracks visual attention and output image correlations to improve visual grounding.
Outcome:	The proposed model outperforms existing approaches and significantly reduces hallucinations.

Similar Papers

Overview

Challenge

Approach

Outcome

Joint Multimodal Preference Optimization for Fine-Grained Visual-Textual Alignment (2026.findings-eacl)

Copied to clipboard

Jiwon Kim, Hyunsoo Yoon

Multimodality

Challenge:	Recent research has focused on addressing multimodal hallucinations in Large Vision-Language Models (LVLMs) however, these methods lack fine-grained visual contrast mechanisms and rely on single-margin optimization.
Approach:	They propose a framework that integrates text-conditioned preference loss with visual ranking-based objective.
Outcome:	The proposed framework improves cross-modal alignment and fine-grained visual grounding.

Re-Align: Aligning Vision Language Models via Retrieval-Augmented Direct Preference Optimization (2025.emnlp-main)

Copied to clipboard

Shuo Xing, Peiran Li, Yuping Wang, Ruizheng Bai, Yueqi Wang, Chan-Wei Hu, Chengxuan Qian, Huaxiu Yao, Zhengzhong Tu

Multimodality

Challenge:	emergence of large Vision Language Models (VLMs) has broadened the capabilities of single-modal Large Language Model (LLM) but VLMs are prone to significant hallucinations, especially in the form of cross-modal inconsistencies.
Approach:	They propose a new alignment framework that leverages image retrieval to integrate both textual and visual preference signals.
Outcome:	The proposed framework mitigates hallucinations more effectively than previous methods . it maintains robustness and scalability across a wide range of VLM sizes and architectures .

Symmetrical Visual Contrastive Optimization: Aligning Vision-Language Models with Minimal Contrastive Images (2025.acl-long)

Copied to clipboard

Shengguang Wu, Fan-Yun Sun, Kaiyue Wen, Nick Haber

Multimodality

Challenge:	Recent studies have shown that Large Vision-Language Models (VLMs) tend to neglect image content and over-rely on language-model priors, resulting in errors in visually grounded tasks and hallucinations.
Approach:	They propose a novel finetuning objective that steers the model toward capturing important visual details and aligning them with corresponding text tokens.
Outcome:	The proposed method achieves up to 22% reduction in hallucinations and significant gains in vision-centric and general tasks while maintaining or improving the model's general abilities.

MoCa: Modality-aware Continual Pre-training Makes Better Bidirectional Multimodal Embeddings (2026.acl-long)

Copied to clipboard

Haonan Chen, Hong Liu, Yuping Luo, Liang Wang, Nan Yang, Furu Wei, Zhicheng Dou

Semantic Text Processing

Challenge:	Existing approaches to embed multimodal models face limitations such as suboptimal causal attention in VLMs and limited diversity in training objectives and data.
Approach:	They propose a framework for transforming pre-trained VLMs into bidirectional multimodal embedding models.
Outcome:	The proposed model improves performance across MMEB and ViDoRe-v2 benchmarks and exhibits strong scalability with both model size and training data on MMEF.

Focus on What Matters: Enhancing Medical Vision-Language Models with Automatic Attention Alignment Tuning (2025.acl-long)

Copied to clipboard

Aofei Chang, Le Huang, Alex James Boyd, Parminder Bhatia, Taha Kass-Hout, Cao Xiao, Fenglong Ma

Language Models

Challenge:	Existing methods rely on inference-time interventions, which are limited in attention adaptation or require additional supervision.
Approach:	They propose a framework for automatic attention alignment tuning that leverages weak labels from SAM and selectively modifies visually-critical attention heads to improve alignment while minimizing interference.
Outcome:	The proposed framework outperforms state-of-the-art models on medical VQA and report generation benchmarks.

RATION: Entropy-Driven Task-Adaptive Visual Attention Allocation Framework for Multimodal Reasoning (2026.findings-acl)

Copied to clipboard

Xingle Xu, Fanheng Kong, Dexian Cai, Shi Feng, Xiaocui Yang, Daling Wang, Yifei Zhang

Multimodality

Challenge:	Prior studies have focused on strengthening multimodal reasoning by improving representation alignment or increasing computation, but these methods do not characterize the differences in visual demands across tasks.
Approach:	They propose an entropy-driven task-adaptive visual attention allocation framework that uses visual attention entropic as a control signal to dynamically allocate attention according to task demands.
Outcome:	The proposed framework achieves consistent performance gains across diverse reasoning tasks, datasets, and models, providing a clear direction toward more reliable multimodal reasoning.

Mask What Matters: Mitigating Object Hallucinations in Multimodal Large Language Models with Object-Aligned Visual Contrastive Decoding (2026.eacl-srw)

Copied to clipboard

Boqi Chen, Xudong Liu, Jianing Qiu

Multimodality

Challenge:	Recent studies improve visual contrastive decoding (VCD) by constructing more informative auxiliary views.
Approach:	They propose to construct an object-aligned auxiliary view that disrupts unsupported tokens and produces a stronger contrast signal.
Outcome:	Empirically, the proposed method shows consistent gains on two popular object hallucination benchmarks across two MLLMs.

Rethinking the Multimodal Correlation of Multimodal Sequential Learning via Generalizable Attentional Results Alignment (2024.acl-long)

Copied to clipboard

Tao Jin, Wang Lin, Ye Wang, Linjun Li, Xize Cheng, Zhou Zhao

Multimodality

Challenge:	Existing studies have focused on the alignment of multimodal sequential learning using transformers.
Approach:	They propose a constrained scheme to align the multiple attentional results from both local and global perspectives.
Outcome:	The proposed scheme could align the multiple attentional results from both local and global perspectives, making the information capture more efficient.

Looking Beyond Text: Reducing Language Bias in Large Vision-Language Models via Multimodal Dual-Attention and Soft-Image Guidance (2025.emnlp-main)

Copied to clipboard

Haozhe Zhao, Shuzheng Si, Liang Chen, Yichi Zhang, Maosong Sun, Baobao Chang, Minjia Zhang

Visual Data in NLP

Challenge:	Large vision-language models (LVLMs) have been criticized for their language bias.
Approach:	They propose to use a dual-attention mechanism to construct separate attention for visual and text inputs to enhance integration of visual inputs across models.
Outcome:	Experiments show that the proposed model debiases LVLMs from their language bias, enhancing visual comprehension and reducing hallucinations without additional resources.

Cat-MoD: Accelerating Multimodal Alignment via Caption Token Guided Asymmetric Mixture-of-Depths (2026.acl-long)

Copied to clipboard

YiJie Huang, Xiaocui Yang, Shi Feng, Wen Zhang, Kaisong Song, Yifei Zhang, Daling Wang

Multimodality

Challenge:	Existing query-based alignment modules enforce uniform cross-attention across all layers, leading to computational redundancy.
Approach:	They propose a framework that allows for asynchronous query-based alignment with large-scale visual features.
Outcome:	The proposed framework matches or surpasses baseline performance while reducing alignment FLOPs by approximately 37% during training and inference.