MoCa: Modality-aware Continual Pre-training Makes Better Bidirectional Multimodal Embeddings (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to embed multimodal models face limitations such as suboptimal causal attention in VLMs and limited diversity in training objectives and data. |
| Approach: | They propose a framework for transforming pre-trained VLMs into bidirectional multimodal embedding models. |
| Outcome: | The proposed model improves performance across MMEB and ViDoRe-v2 benchmarks and exhibits strong scalability with both model size and training data on MMEF. |
Similar Papers
Compressing then Matching: An Efficient Pre-training Paradigm for Multimodal Embedding (2026.acl-long)
Copied to clipboard
Da Li, Yuxiao Luo, Keping Bi, Jiafeng Guo, Wei Yuan, Biao Yang, Yan Wang, Fan Yang, Tingting Gao, Guorui Zhou
| Challenge: | Recent approaches demonstrate that MLLMs can be adapted into competitive embedding models via large-scale contrastive learning. |
| Approach: | They propose a compressed pre-training phase which serves as a warm-up stage for contrastive learning. |
| Outcome: | The proposed model achieves state-of-the-art among MLLMs of comparable size on the MMEB, realizing optimization in both efficiency and effectiveness. |
VisiPruner: Decoding Discontinuous Cross-Modal Dynamics for Efficient Multimodal LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Multimodal Large Language Models (MLLMs) suffer from significant computational overhead due to the quadratic growth of attention computations with the number of multimodal tokens. |
| Approach: | They propose a training-free pruning framework that prunes multimodal tokens without a trained pruning method. |
| Outcome: | The proposed pruning framework outperforms existing token pruning methods and generalizes across diverse MLLMs. |
Merge then Realign: Simple and Effective Modality-Incremental Continual Learning for Multimodal LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in Multimodal Large Language Models (MLLMs) have enhanced their versatility as they integrate a growing number of modalities. |
| Approach: | They propose a simple MCL paradigm that addresses forgetting and misalignment . they propose 'MErge then ReAlign' to extend existing models to more modalities . |
| Outcome: | The proposed paradigm is easy to deploy and highly reusable in the MLLM community. |
Multimodal Unlearning Across Vision, Language, Video, and Audio: Survey of Methods, Datasets, and Benchmarks (2026.findings-acl)
Copied to clipboard
| Challenge: | Survey aims to identify challenges of multimodal unlearning for vision, language, audio and video . retraining after deletion requests or policy updates is often impractical, survey finds . |
| Approach: | They propose to enable selective removal across modalities while retaining overall utility. |
| Outcome: | This study compares models with existing models to identify weaknesses and improves performance. |
Two Challenges, One Solution: Robust Multimodal Learning through Dynamic Modality Recognition and Enhancement (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods require full-modality data during training phase or require explicit annotations to detect missing modalities. |
| Approach: | They propose a Dynamic modality Recognition and Enhancement for Adaptive Multimodal fusion framework that directs selective reconstruction of missing or underperforming modalities. |
| Outcome: | The proposed framework outperforms several baseline and state-of-the-art models on three benchmark datasets. |
Causal2Vec: Improving Decoder-only LLMs as Embedding Models through a Contextual Token (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods modify attention mechanism to be bidirectional, undermining LLMs’ ability to extract semantic information acquired during pre-training. |
| Approach: | They propose a general-purpose embedding model that pre-encodes input text into a single Contextual token and then prepends it to the LLM's input sequence. |
| Outcome: | The proposed model improves performance of decoder-only large language models without altering their architectures or introducing significant computational overhead. |
MMEvol: Empowering Multimodal Large Language Models with Evol-Instruct (2025.findings-acl)
Copied to clipboard
Run Luo, Haonan Zhang, Longze Chen, Ting-En Lin, Xiong Liu, Yuchuan Wu, Min Yang, Yongbin Li, Minzheng Wang, Pengpeng Zeng, Lianli Gao, Heng Tao Shen, Yunshui Li, Hamid Alinejad-Rokny, Xiaobo Xia, Jingkuan Song, Fei Huang
| Challenge: | a new framework for image-text instruction data evolution improves MLLM performance . lack of high-quality instruction data remains a major bottleneck in ML modeling . |
| Approach: | They propose a multimodal instruction data evolution framework that iteratively enhances data quality through fine-grained perception, cognitive reasoning, and interaction evolution. |
| Outcome: | The proposed approach improves MLLM performance in nine vision-language tasks while using significantly less data. |
AFMRL: Attribute-Enhanced Fine-Grained Multi-Modal Representation Learning in E-commerce (2026.findings-acl)
Copied to clipboard
| Challenge: | Multimodal representation is crucial for E-commerce tasks such as identical product retrieval. |
| Approach: | They propose an approach which leverages the generative power of Multimodal Large Language Models to extract key attributes from product images and text and enhances representation learning through a two-stage training framework. |
| Outcome: | The proposed model achieves state-of-the-art on multiple downstream retrieval tasks, validating the effectiveness of harnessing generative models to advance fine-grained representation learning. |
Vision-Language Pretraining: Current Trends and the Future (2022.acl-tutorials)
Copied to clipboard
| Challenge: | Recent vision-language models are being used for downstream tasks that require large datasets and supervised datasets. |
| Approach: | They focus on recent vision-language pretraining paradigms and their strengths and shortcomings . they compare the different family of models used for vision- language pretraining . |
| Outcome: | This paper provides the background on image–language datasets, benchmarks, and modeling innovations before the multimodal pretraining area. |
Unifying Cross-Lingual and Cross-Modal Modeling Towards Weakly Supervised Multilingual Vision-Language Pre-training (2023.acl-long)
Copied to clipboard
| Challenge: | Existing studies address the problem of translating English data into other languages, but they are limited in form and scale. |
| Approach: | They propose a framework to unify cross-lingual and cross-modal pre-training by using English data. |
| Outcome: | The proposed framework unifies cross-lingual and cross-modal pre-training on different data. |