Vision-aided Unsupervised Constituency Parsing with Multi-MLLM Debating (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches require explicit cross-modal alignment, but new approaches address these challenges. |
| Approach: | They propose a framework for vision-aided unsupervised constituency parsing . they leverage multimodal large language models pre-trained on diverse image-text or video-text data . |
| Outcome: | The proposed framework achieves state-of-the-art performance on image-text and video-text datasets, improving robustness and accuracy. |
Similar Papers
Multimodal Large Language Models for Text-rich Image Understanding: A Comprehensive Review (2025.findings-acl)
Copied to clipboard
Pei Fu, Tongkun Guan, Zining Wang, Zhentao Guo, Chen Duan, Hao Sun, Boming Chen, Qianyi Jiang, Jiayao Ma, Kai Zhou, Junfeng Luo
| Challenge: | Recent advances in vision-language models have unified perception and understanding tasks within Visual Question Answering paradigms. |
| Approach: | They propose to outline timeline, architecture, and pipeline of nearly all TIU MLLMs and review their performance on mainstream benchmarks. |
| Outcome: | The proposed models perform well on mainstream benchmarks and are compared with other models. |
Unifying Text, Tables, and Images for Multimodal Question Answering (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to multimodal question answering rely on single-modal or bi-modal models, which limit their ability to integrate information across all modalities. |
| Approach: | They propose a framework that unifies three different input modalities into a text-to-text format by employing position-enhanced table linearization and diversified image captioning techniques. |
| Outcome: | The proposed framework unifies three input modalities into a text-to-text format using position-enhanced table linearization and diversified image captioning techniques. |
The Revolution of Multimodal Large Language Models: A Survey (2024.findings-acl)
Copied to clipboard
Davide Caffagni, Federico Cocchi, Luca Barsellotti, Nicholas Moratelli, Sara Sarto, Lorenzo Baraldi, Lorenzo Baraldi, Marcella Cornia, Rita Cucchiara
| Challenge: | Recent advances in large language models have led to the development of multimodal large language model. |
| Approach: | They present a review of recent visual-based Large Language Models and analyze their architectures and alignment strategies. |
| Outcome: | The proposed models can integrate visual and textual modalities while providing a dialogue-based interface and instruction-following capabilities. |
When Big Models Train Small Ones: Label-Free Model Parity Alignment for Efficient Visual Question Answering using Small VLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large vision and language models have demonstrated remarkable performance in visual question answering tasks. |
| Approach: | They introduce a framework to optimize L-VLMs by leveraging unlabeled images . they conduct extensive experiments on four diverse VQA benchmarks . |
| Outcome: | The proposed framework improves L-VLMs on four visual question answering benchmarks. |
Debating for Better Reasoning in Vision-Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) gain expertise across diverse domains and modalities, a new study shows . scalable oversight becomes challenging when their capabilities surpass human evaluators. |
| Approach: | a new study extends the debate paradigm to a multimodal setting . it explores the potential for blind models to supervise and enhance the performance of sighted ones. |
| Outcome: | The proposed framework outperforms individual LLMs on multimodal tasks . it allows blind models to supervise and enhance the performance of sighted models . |
KG-ViP: Bridging Knowledge Grounding and Visual Perception in Multi-modal LLMs for Visual Question Answering (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to Visual Question Answering lack synergistic potential of scene graphs and scene graph. |
| Approach: | They propose a retrieval-and-fusion pipeline that fuses scene graphs and commonsense graphs to enable multi-modal reasoning. |
| Outcome: | Experiments on FVQA 2.0+ and MVQA benchmarks show that KG-ViP outperforms existing methods. |
A Survey on MLLM-based Visually Rich Document Understanding: Methods, Challenges, and Emerging Trends (2026.findings-acl)
Copied to clipboard
Yihao Ding, Siwen Luo, Yue Dai, Yanbei Jiang, Zechuan Li, Qiang Sun, Geoffrey Martin, Wei Liu, Yifan Peng
| Challenge: | Visually Rich Document Understanding (VRDU) frameworks are a key area of research . early approaches to VRDU relied on manually crafted rules and domain-specific heuristics . conventional deep learning approaches do not integrate the diverse modalities in documents . |
| Approach: | They review recent advances in MLLM-based Visually Rich Document Understanding (VRDU) their findings highlight emerging trends and promising research directions . |
| Outcome: | The proposed frameworks are scalable, reliable, and adaptable, the authors argue . their findings highlight emerging trends and promising research directions . |
AlignMMBench: Evaluating Chinese Multimodal Alignment in Large Vision-Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks focus on basic abilities using nonverbal methods, such as yes-no and multiple-choice questions. |
| Approach: | They propose a benchmark that provides more nuanced evaluations of alignment capabilities for large Vision-Language Models (VLMs) they use a rule-calibrated evaluator that exceeds GPT-4's evaluation ability and a “alignment score” to assess the robustness and stability of models across diverse prompts. |
| Outcome: | The proposed benchmark covers 13 tasks across three categories and includes both single-turn and multi-turn dialogue scenarios. |
Towards Unified Multimodal Large Language Models: A survey (2026.findings-acl)
Copied to clipboard
| Challenge: | unified multimodal large language models (MLLMs) are emerging but lack a systematic framework to connect them and situate current trends within a broader landscape. |
| Approach: | They present a systematic review of unified Multimodal Large Language Models . they outline the foundational concepts and prerequisites for understanding them . |
| Outcome: | The present review provides a systematic and systematic overview of unified MLLMs . it discusses persistent challenges and identify promising directions for future research . |
SEA: Supervised Embedding Alignment for Token-Level Visual-Textual Integration in MLLMs (2025.emnlp-main)
Copied to clipboard
Yuanyang Yin, Yaqi Zhao, Yajie Zhang, Yuanxing Zhang, Ke Lin, Jiahao Wang, Xin Tao, Pengfei Wan, Wentao Zhang, Feng Zhao
| Challenge: | Multimodal Large Language Models (MLLMs) integrate visual and textual inputs, yet modality alignment remains one of the most challenging aspects. |
| Approach: | They propose a token-level supervision alignment method that enables more precise visual-text alignment during pretraining. |
| Outcome: | The proposed method improves performance across various model sizes, with smaller models benefiting the most. |