VLM

ConECT Dataset: Overcoming Data Scarcity in Context-Aware E-Commerce MT (2025.acl-short)

Challenge:	Existing methods for document understanding are memory-intensive and impractical for local deployments.
Approach:	They propose a document navigation expert that leverages a Chain of Scroll mechanism to selectively and recursively navigate documents, focusing exclusively on relevant segments.
Outcome:	The proposed method reduces memory usage and effectively models human-like reading behaviors.

Copied to clipboard

Mikołaj Pokrywka, Wojciech Kusa, Mieszko Rutkowski, Mikołaj Koszowski

Machine Translation

Challenge:	Neural Machine Translation (NMT) has improved translation by using Transformer-based models, but still struggles with word ambiguity and context.
Approach:	They create a new Czech-to-polish e-commerce product translation dataset coupled with images and product metadata consisting of 11,400 sentence pairs.
Outcome:	The proposed model incorporates visual cues alongside textual data to improve translation quality.

Router-Suggest: Dynamic Routing for Multimodal Auto-Completion in Visually-Grounded Dialogs (2026.eacl-industry)

Copied to clipboard

Sandeep Mishra, Devichand Budagam, Anubhab Mandal, Bishal Santra, Pawan Goyal, Manish Gupta

LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkit (2024.emnlp-industry)

Challenge:	a task that grounds predictions in multimodal context is essential for chatbots, chatbot systems and healthcare consultations.
Approach:	They propose a task that grounds predictions in multimodal context to better capture user intent.
Outcome:	The proposed task can be used to predict upcoming characters in live chats using partially typed text and visual cues.

Copied to clipboard

Ruihao Gong, Yang Yong, Shiqiao Gu, Yushi Huang, Chengtao Lv, Yunchen Zhang, Dacheng Tao, Xianglong Liu

Dash-M5H: An Interactive Dashboard for Multi-Modal, Multi-Model Mental Health Assessment (2026.acl-demo)

Challenge:	Existing quantization techniques have been categorized as 'simple' and 'highly efficient' however, their configurations vary from each other and cannot be fairly compared .
Approach:	They propose a plug-and-play compression toolkit to explore the impact of quantization.
Outcome:	The proposed toolkit explores the impact of quantization on large language models.

Copied to clipboard

Raymond Alavo, Xinyuan Zhang, Gemza Ademaj, Junhui Cai, Hyeokhyen Kwon, Robert Cotes, Gari D. Clifford, Ahmed Abbasi

BattleAgent: Multi-modal Dynamic Emulation on Historical Battles to Complement Historical Analysis (2024.emnlp-demo)

Challenge:	Dash-M5H integrates transcript text, audio, and facial behavior with a clinically grounded VLM prediction pipeline that produces DSM-5-aligned depression predictions.
Approach:	They propose a dashboard that integrates multimodal behavioral data with multi-model signal outputs of recorded clinical interviews.
Outcome:	Dash-M5H is an interactive dashboard for *multi-modal, multi-model mental health assessment that integrates transcript text, audio, and facial behavior with a clinically grounded VLM prediction pipeline.

Copied to clipboard

Shuhang Lin, Wenyue Hua, Lingyao Li, Che-Jui Chang, Lizhou Fan, Jianchao Ji, Hang Hua, Mingyu Jin, Jiebo Luo, Yongfeng Zhang

UNIVID: Unified Vision-Language Model for Video Moderation (2026.acl-industry)

Challenge:	Recent advances in large language models have demonstrated impressive reasoning capabilities, indicating their potential to serve as the foundation for agents.
Approach:	They propose a detailed emulation system that combines large vision-language model and multi-agent system to emulate dynamic interactions between multiple agents over a period of time.
Outcome:	The proposed system combines large vision-language model and multi-agent system to emulate dynamic interactions between agents and their environments over a period of time.

Copied to clipboard

Kejuan Yang, Yizhuo Zhang, Mingyuan Du, Yue Zhang, Dixin Zheng, Kaili Zhao, Yang Xiao, Hanzhong Liang, Kenan Xiao

MoCa: Modality-aware Continual Pre-training Makes Better Bidirectional Multimodal Embeddings (2026.acl-long)

Challenge:	Existing video moderation systems rely on fragmented black-box classification models that are difficult to maintain and lack transparency.
Approach:	They propose a Unified Vision-Language model for Video Moderation that generates policy-aware captions that serve as an interpretable intermediate representation.
Outcome:	The proposed model reduces violation leakage and overkill rate by 42.7% while reducing maintenance costs.

Copied to clipboard

Haonan Chen, Hong Liu, Yuping Luo, Liang Wang, Nan Yang, Furu Wei, Zhicheng Dou

Do Vision-Language Models Understand Compound Nouns? (2024.naacl-short)

Challenge:	Existing approaches to embed multimodal models face limitations such as suboptimal causal attention in VLMs and limited diversity in training objectives and data.
Approach:	They propose a framework for transforming pre-trained VLMs into bidirectional multimodal embedding models.
Outcome:	The proposed model improves performance across MMEB and ViDoRe-v2 benchmarks and exhibits strong scalability with both model size and training data on MMEF.

Copied to clipboard

Sonal Kumar, Sreyan Ghosh, S Sakshi, Utkarsh Tyagi, Dinesh Manocha

AutoTrain: No-code training for state-of-the-art models (2024.emnlp-demo)

Challenge:	Open-vocabulary vision-language models (CLIP) are emerging as a promising new paradigm for text-to-image retrieval.
Approach:	They propose a benchmark to evaluate the effectiveness of open-vocabulary vision-language models (CLIP) for text-to-image retrieval using contrastive loss.
Outcome:	The proposed framework improves CN understanding of CLIP by 8.25% on Compun.

Copied to clipboard

Abhishek Thakur

DASR: Distributed Adaptive Scene Recognition - A Multi-Agent Cloud-Edge Framework for Language-Guided Scene Detection (2025.emnlp-industry)

Challenge:	AutoTrain is an open-source, no code tool/library which can be used to train models on custom datasets.
Approach:	They propose an open-source, no-code tool/library to train models on custom datasets.
Outcome:	The open-source, no-code tool/library can be used to train models on custom datasets.

Copied to clipboard

Can Cui, Yongkang Liu, Seyhan Ucar, Juntong Peng, Ahmadreza Moradipari, Maryam Khabazi, Ziran Wang

Spectra: A Mechanistic Interpretability Library for Vision-Language Models (2026.acl-demo)

Challenge:	Current approaches to analyzing driving scenarios rely on massive data collection followed by manual filtering.
Approach:	They propose a cloud-based framework for language-guided scene detection in connected vehicles . the framework leverages cloud- and edge-deployed large language models to identify relevant driving scenarios while optimizing on-vehicle buffer storage.
Outcome:	The proposed framework performs better on complex driving tasks and reduces storage requirements.

Copied to clipboard

Clement Neo, Yongsen Zheng, Kwok-Yan Lam, Luke Ong

Explainability & Interpretability in NLP

Challenge:	Existing interpretability tools for visionlanguage models are limited to activation probing and saving.
Approach:	They propose a library specifically designed for mechanistic interpretability of visionlanguage models that provides unified abstractions for activation patching, attention pattern analysis, and meta-functions across diverse VLM architectures.
Outcome:	The proposed library handles architecture-specific complexities while maintaining a simple, high-level interface.

Act as you think: Reinforcing Consistent Reasoning in Medical Visual Question Answering (2026.acl-long)

Copied to clipboard

Songtao Jiang, Yuan Wang, Ruizhe Chen, Yan Zhang, Ruilin Luo, Bohan Lei, Yeying Jin, Sibo Song, ZhiBo Yang, Jimeng Sun, Jian Wu, Zuozhu Liu

Improve Vision Language Model Chain-of-thought Reasoning (2025.acl-long)

Challenge:	Recent advances have improved the accuracy of medical visual question answering (Med-VQA) however, the high stakes nature of the medical domain has precipitated a shift towards interpretability and transparency of reasoning processes.
Approach:	They propose a reinforcement learning from verifiable rewards framework that rewards internal consistency and logical coherence.
Outcome:	The proposed framework rewards internal consistency and logical coherence, and is highly versatile, the authors show.

Copied to clipboard

Ruohong Zhang, Bowen Zhang, Yanghao Li, Haotian Zhang, Zhiqing Sun, Zhe Gan, Yinfei Yang, Ruoming Pang, Yiming Yang

MATE: Meet At The Embedding - Connecting Images with Long Texts (2024.findings-emnlp)

Challenge:	Current training recipes often rely on datasets dominated by short annotations with limited rationales, hindering the models' ability to generalize to tasks requiring comprehensive reasoning.
Approach:	They propose a two-stage post-training strategy that augments short answers with CoT reasoning generated by GPT-4o, enhancing the VLM's CoT capabilities through fine-tuning.
Outcome:	The proposed strategy enhances the model's CoT capabilities through fine-tuning and reinforcement learning.

Copied to clipboard

Young Kyun Jang, Junmo Kang, Yong Jae Lee, Donghyun Kim

Cache-of-Thought: Master-Apprentice Framework for Cost-Effective Vision Language Model Reasoning (2025.emnlp-main)

Challenge:	Recent advances in Vision Language Models (VLMs) focus on aligning images with short descriptive captions.
Approach:	They propose a method that combines VLMs with Large Language Models to efficiently align images with long texts without additional text pairs.
Outcome:	The proposed method bridges the gap between VLM and LLM without additional image-long text pairs.

Copied to clipboard

Mingyuan Wu, Jize Jiang, Haozhen Zheng, Meitang Li, Zhaoheng Li, Beitong Tian, Bo Chen, Yongjoo Park, Minjia Zhang, ChengXiang Zhai, Klara Nahrstedt

Data or Language Supervision: What Makes CLIP Better than DINO? (2025.findings-emnlp)

Challenge:	Recent Vision Language Models (VLMs) have shown tremendous promise in a wide range of realworld applications, but their size has made at-scale deployment and operation challenging due to high consumption of cloud computing resource, high latency, and expensive API calls.
Approach:	They propose a master–apprentice framework for collaborative inference between large and small vision language models.
Outcome:	The proposed framework improves reasoning performance on widely-recognized and challenging general reasoning benchmarks and specifically boosts reasoning of apprentice VLMs by 36.6%.

Copied to clipboard

Yiming Liu, Yuhui Zhang, Dhruba Ghosh, Ludwig Schmidt, Serena Yeung-Levy

ColorSwap: A Color and Word Order Dataset for Multimodal Evaluation (2024.findings-acl)

Challenge:	CLIP outperforms self-supervised models like DINO as vision encoders for vision-language models (VLMs) but it remains unclear whether this advantage stems from CLIP’s language supervision or its much larger training data.
Approach:	Embedding analysis shows CLIP captures high-level semantics while DINO is more responsive to low-level features like colors and styles.
Outcome:	Embedding analysis shows that CLIP captures high-level semantics, while DINO is more responsive to low-level features like colors and styles.

Copied to clipboard

Jirayu Burapacheep, Ishan Gaur, Agam Bhatia, Tristan Thrush

CARES: Context-Aware Resolution Selector for VLMs (2026.acl-long)

Challenge:	Recent work reveals that vision and language models struggle to comprehend fine grained distinctions in images.
Approach:	They propose a dataset to assess multimodal models' ability to match objects with their colors.
Outcome:	The proposed model performs well in visual questionanswering, text-to-image generation and word-order understanding tasks.

Copied to clipboard

Moshe Kimhi, Nimrod Shabtay, Raja Giryes, Chaim Baskin, Eli Schwartz

Enhancing Fine-Grained Image Classifications via Cascaded Vision Language Models (2024.findings-emnlp)

Challenge:	Large vision–language models process images at native or high resolution to remain effective across tasks.
Approach:	They propose a lightweight preprocessing module that predicts the minimum sufficient input resolution for large vision–language models.
Outcome:	CARES predicts when a pre-trained VLM's response converges to its peak ability to answer correctly, reducing compute by up to 80%.

Copied to clipboard

Canshi Wei

Information Retrieval

Challenge:	Fine-grained image classification is a challenge for vision-language models (VLMs) such as CLIP, which struggle to distinguish between semantically similar classes due to insufficient supervision for fine-grain tasks.
Approach:	They propose a framework that harnesses the complementary strengths of both CLIP-like and LVLMs to tackle these challenges.
Outcome:	The proposed framework outperforms existing models on multiple fine-grained datasets, particularly the Stanford Cars dataset.

A Structured Framework for Evaluating and Enhancing Interpretive Capabilities of Multimodal LLMs in Culturally Situated Tasks (2025.findings-emnlp)

Copied to clipboard

Haorui Yu, Ramon Ruiz-Dolz, Qiufeng Yi

Re-Align: Aligning Vision Language Models via Retrieval-Augmented Direct Preference Optimization (2025.emnlp-main)

Challenge:	Using a zero-shot classification model, we extracted multi-dimensional evaluative features from human expert critiques and used them to evaluate selected VLMs such as Llama, Qwen, or Gemini.
Approach:	They constructed a quantitative framework for Chinese painting critique by extracting multi-dimensional evaluative features from human expert critiques using a zero-shot classification model.
Outcome:	The framework was constructed by extracting features from human critiques using a zero-shot classification model.

Copied to clipboard

Shuo Xing, Peiran Li, Yuping Wang, Ruizheng Bai, Yueqi Wang, Chan-Wei Hu, Chengxuan Qian, Huaxiu Yao, Zhengzhong Tu

Deploying Tiny LVLM Judges for Real-World Evaluation of Chart Models: Lessons Learned and Best Practices (2025.emnlp-industry)

Challenge:	emergence of large Vision Language Models (VLMs) has broadened the capabilities of single-modal Large Language Model (LLM) but VLMs are prone to significant hallucinations, especially in the form of cross-modal inconsistencies.
Approach:	They propose a new alignment framework that leverages image retrieval to integrate both textual and visual preference signals.
Outcome:	The proposed framework mitigates hallucinations more effectively than previous methods . it maintains robustness and scalability across a wide range of VLM sizes and architectures .

Copied to clipboard

Md Tahmid Rahman Laskar, Mohammed Saidul Islam, Ridwan Mahbub, Mizanur Rahman, Amran Bhuiyan, Israt Jahan, Mir Tafseer Nayeem, Shafiq Joty, Enamul Hoque, Jimmy Huang

COCO-Tree: Compositional Hierarchical Concept Trees for Enhanced Reasoning in Vision-Language Models (2025.emnlp-main)

Challenge:	Large Vision-Language Models (LVLMs) with only 7B parameters perform poorly as judges in resource-constrained settings.
Approach:	They propose two approaches to ensure costefficient evaluation by combining multiple criteria into a single query and domainadaptive transfer learning to create a 2Bparameter VLM on a chart dataset.
Outcome:	The proposed model can effectively transfer knowledge from one dataset to another to make it a more specialized model.

Copied to clipboard

Sanchit Sinha, Guangzhi Xiong, Aidong Zhang

Graph-guided Cross-composition Feature Disentanglement for Compositional Zero-shot Learning (2025.findings-acl)

Challenge:	Existing approaches to improve compositional reasoning in vision language models are resource-intensive or do not provide an interpretable reasoning process.
Approach:	They propose a method that augments VLM outputs with carefully designed neurosymbolic concept trees learned from LLMs to improve VLM’s linguistic reasoning.
Outcome:	Empirical results show that COCO-Tree significantly improves compositional generalization and provides a rationale behind VLM predictions.

Copied to clipboard

Yuxia Geng, Runkai Zhu, Jiaoyan Chen, Jintai Chen, Xiang Chen, Zhuo Chen, Shuofei Qiao, Yuxiang Wang, Xiaoliang Xu, Sheng-Jun Huang

Automating eHMI Action Design with LLMs for Automated Vehicle Communication (2025.findings-emnlp)

Challenge:	Disentanglement of visual features of primitives (i.e., attributes and objects) has shown exceptional results in Compositional Zero-shot Learning (CZSL).
Approach:	They propose a solution that takes multiple compositions as inputs and constrains disentangled primitive features to be general across compositions.
Outcome:	The proposed architecture significantly improves performance on three popular CZSL benchmarks and has been verified by solid ablation studies.

Copied to clipboard

Ding Xia, Xinyue Gui, Fan Gao, Dongyuan Li, Mark Colley, Takeo Igarashi

Vision-Language Models Align with Human Neural Representations in Concept Processing (2026.eacl-long)

Challenge:	Currently, eHMIs employ predefined text messages and manually designed actions to perform these messages . this limits the real-world deployment of ehMIs, where adaptability in dynamic scenarios is essential.
Approach:	They propose a pipeline that integrates large language models and 3D renderers to generate executable actions for controlling eHMIs and rendering action clips.
Outcome:	The proposed pipeline integrates large language models and 3D renderers to generate executable actions for controlling eHMIs and rendering action clips.

Copied to clipboard

Anna Bavaresco, Marianne De Heer Kloots, Sandro Pezzelle, Raquel Fernández

Image Embedding Sampling Method for Diverse Captioning (2025.emnlp-main)

Challenge:	Recent studies suggest that transformer-based vision-language models capture the multimodality of concept processing in the human brain.
Approach:	They analysed multiple VLMs employing different strategies to integrate visual and textual modalities, along with language-only counterparts.
Outcome:	The transformer-based vision-language models outperform language-only models in two experimental conditions, while only some outperformed the language-based models.

Copied to clipboard

Sania Waheed, Na Min An

Addressing Blind Guessing: Calibration of Selection Bias in Multiple-Choice Question Answering by Video Language Models (2025.acl-long)

Challenge:	Currently, large-scale captioning models are less accessible for resource-constrained applications such as mobile devices and assistive technologies.
Approach:	They propose a training-free framework that enhances caption diversity and informativeness by explicitly attending to distinct image regions using a comparably small VLM as the backbone.
Outcome:	The proposed framework achieves comparable performance to larger models on MSCOCO, Flickr30k, and Nocaps test datasets while maintaining strong image-caption relevancy and semantic integrity with the human-annotated captions.

Copied to clipboard

Olga Loginova, Oleksandr Bezrukov, Ravi Shekhar, Alexey Kravets

NegVQA: Can Vision Language Models Understand Negation? (2025.findings-acl)

Challenge:	Existing MCQA benchmarks fail to capture the full reasoning capabilities of video language models due to selection bias.
Approach:	They propose a method to reduce selection bias in video-to-text LLMs by suppressing "blind guessing" they propose 'bold' calibration technique to balance selection bias.
Outcome:	The proposed method reduces selection bias and improves model performance compared to existing methods.

Copied to clipboard

Yuhui Zhang, Yuchang Su, Yiming Liu, Serena Yeung-Levy

Journey Before Destination: On the importance of Visual Faithfulness in Slow Thinking (2026.eacl-long)

Challenge:	NegVQA is a visual question answering (VQA) benchmark consisting of 7,379 two-choice questions covering diverse negation scenarios and image-question distributions.
Approach:	They propose a visual question answering benchmark consisting of 7,379 two-choice questions covering diverse negation scenarios and image-question distributions.
Outcome:	The proposed model fails to correctly interpret negation, leading to critical errors in interactive AI systems.

Copied to clipboard

Rheeya Uppaal, Phu Mon Htut, Min Bai, Nikolaos Pappas, Zheng Qi, Sandesh Swamy

AVA: Attentive VLM Agent for Mastering StarCraft II (2026.findings-acl)

Challenge:	Existing evaluations for visual hallucinations are narrow.
Approach:	They propose a framework that decomposes reasoning chains into perception versus reasoning steps and uses off-the-shelf VLM judges for step-level faithfulness.
Outcome:	The proposed framework reduces Unfaithful Perception Rate while preserving final-answer accuracy.

Copied to clipboard

Weiyu Ma, Yuqian Fu, Zecheng Zhang, Bernard Ghanem, Guohao Li

Decompose and Compare Consistency: Measuring VLMs’ Answer Reliability via Task-Decomposition Consistency Comparison (2024.emnlp-main)

Challenge:	Existing StarCraft II benchmarks rely on abstract state representations that deviate from human perception . Existing systems rely only on abstract representations, creating an artificial gap between how humans process battlefield information and limiting ecological validity of learned behaviors.
Approach:	They introduce AVACraft, the first multimodal benchmark environment for complex decision-making in StarCraft II.
Outcome:	The AVACraft benchmark supports both traditional and modern multi-agent reinforcement learning paradigms.

Copied to clipboard

Qian Yang, Weixiang Yan, Aishwarya Agrawal

BLEnD-Vis: Benchmarking Multimodal Cultural Understanding in Vision Language Models (2026.eacl-long)

Challenge:	Existing methods for estimating uncertainty using answer likelihoods or prompt-based confidence generation often suffer from overconfidence and confirmation biases.
Approach:	They propose to use Decompose and Compare Consistency (DeCC) to measure the reliability of a VLM's direct answer and indirect answers by decomposing the question into sub-questions and reasoning over the sub-answers.
Outcome:	Experiments on six vision-language tasks with three VLMs show that DeCC achieves better correlation with task accuracy compared to existing methods.

Copied to clipboard

Bryan Chen Zhengyu Tan, Weihua Zheng, Zhengyuan Liu, Nancy F. Chen, Hwaran Lee, Kenny Tsu Wei Choo, Roy Ka-Wei Lee

Look Less, Reason More: Rollout-Guided Adaptive Pixel-Space Reasoning (2026.acl-long)

Challenge:	Existing evaluations assess static recall or isolated visual grounding, leaving unanswered whether VLMs possess robust and transferable cultural understanding.
Approach:	They propose a multimodal, multicultural benchmark to evaluate the robustness of everyday cultural knowledge in vision-language models across linguistic rephrasings and visual modalities.
Outcome:	‘BLEnD-Vis‘ constructs 313 culturally grounded question templates spanning 16 regions and generates three aligned multiple-choice formats.

Copied to clipboard

Xuchen Li, Xuzhao Li, Jiahui Gao, Renjie Pi, Shiyu Hu, Wentao Zhang

Do Not Guess, Verify: Logic-Guided Adaptive Reasoning for Multimodal Misinformation Detection (2026.findings-acl)

Challenge:	Recent work has shown promise by incorporating pixel-level visual information into the reasoning process, enabling VLMs to access high-resolution visual details during their thought process.
Approach:	They propose a framework that dynamically determines necessary pixel-level operations based on the input query.
Outcome:	The proposed model achieves 73.4% accuracy on HR-Bench 4K while maintaining a tool usage ratio of only 20.1%, improving accuracy and reducing tool usage by 66.5% compared to the previous methods.

Copied to clipboard

Kun Huang, Rui Qiu, Xiaoming Li, Salah Uddin

Reasoning

Challenge:	Existing multimodal misinformation detection paradigms rely on passive aggregation of multimodal features and social signals.
Approach:	They propose a verification-oriented framework that integrates large vision–language models into multimodal misinformation detection through explicit rationale-guided reasoning.
Outcome:	The proposed framework outperforms state-of-the-art methods on multimodal misinformation detection benchmarks while significantly reducing computational cost.

MM-R3: On (In-)Consistency of Vision-Language Models (VLMs) (2025.findings-acl)

Copied to clipboard

Shih-Han Chou, Shivam Chandhok, Jim Little, Leonid Sigal

Your Vision-Language Model Itself Is a Strong Filter: Towards High-Quality Instruction Tuning with Data Selection (2024.findings-acl)

Challenge:	a flurry of research has been conducted on the performance of state-of-the-art (SoTA) Vision Language Models (VLMs) on a variety of tasks.
Approach:	They propose a benchmarking tool to analyze performance of SoTA Vision Language Models (VLMs) on three tasks: Question Rephrasing, Image Restyling, and Context Reasoning.
Outcome:	The proposed model achieves absolute improvements of 5.7% and 12.5% on widely used VLMs such as BLIP-2 and LLaVa 1.5M in terms of consistency over their existing counterparts.

Copied to clipboard

Ruibo Chen, Yihan Wu, Lichang Chen, Guodong Liu, Qi He, Tianyi Xiong, Chenxi Liu, Junfeng Guo, Heng Huang

FakeSV-VLM: Taming VLM for Detecting Fake Short-Video News via Progressive Mixture-Of-Experts Adapter (2025.findings-emnlp)

Challenge:	Existing data selection methods for instruction-following large language models rely on unreliable scores or use downstream tasks for selection.
Approach:	They propose a method that utilizes the VLM itself as a filter to select high-quality instruction-tuning data.
Outcome:	The proposed method can reach better results compared to full data settings with merely about 15% samples and can achieve superior performance against competitive baselines.

Copied to clipboard

JunXi Wang, Yaxiong Wang, Lechao Cheng, Zhun Zhong

AdaV: Adaptive Text-visual Redirection for Vision-Language Models (2025.findings-acl)

Challenge:	Existing methods for detecting fake news videos fall short due to lack of knowledge to verify the news is real or not.
Approach:	They propose a VLM-based framework for detecting fake news on short video platforms . they design four experts tailored to handle each scenario and integrate them into VLM .
Outcome:	The proposed framework outperforms current state-of-the-art models on two benchmark datasets.

Copied to clipboard

Jiayi Han, Liang Du, Yiwen Wu, Guanming Liang, Xiangguo Zhou, Weibo Zheng, Donghong Han, Zixun Sun

World to Code: Multi-modal Data Generation via Self-Instructed Compositional Captioning and Filtering (2024.emnlp-main)

Challenge:	Vision-language models often generate excessive visual tokens, leading to poor performance . a novel training-free visual token pruning method is proposed to improve performance despite the computational cost associated with VLMs.
Approach:	They propose a training-free visual token pruning method that reduces biased token pruning . they plan to open-source the code upon publication .
Outcome:	The proposed method reduces biased token pruning and enhances model robustness with limited visual token budget.

Copied to clipboard

Jiacong Wang, Bohong Wu, Haiyong Jiang, Zhou Xun, Xin Xiao, Haoyuan Guo, Jun Xiao

WikiSeeker: Rethinking the Role of Vision-Language Models in Knowledge-Based Visual Question Answering (2026.findings-acl)

Challenge:	Recent advances in Vision-Language Models and the scarcity of high-quality multi-modal alignment data have inspired numerous researches on synthetic VLM data generation.
Approach:	They propose a multi-modal data construction pipeline that organizes the final output into a Python code format.
Outcome:	The proposed pipeline improves visual question answering and visual grounding benchmarks across different VLMs.

Copied to clipboard

Yingjian Zhu, Xinming Wang, Kun Ding, Ying Wang, Bin Fan, Shiming Xiang

Don’t Click That: Teaching Web Agents to Resist Deceptive Interfaces (2026.acl-long)

Challenge:	Existing methods for Knowledge-Based Visual Question Answering rely on images as the retrieval key, and often overlook or misplace the role of Vision-Language Models (VLMs)
Approach:	They propose a multi-modal RAG framework that assigns VLMs two specialized agents: a Refiner and an Inspector.
Outcome:	Experiments on EVQA, InfoSeek, and M2KR show that the proposed framework achieves state-of-the-art performance with significant improvements in both retrieval accuracy and answer quality.

Copied to clipboard

Yilin Zhang, Yingkai Hua, Chunyu Wei, Xin Wang, Yueguo Chen

Challenge:	Existing approaches to deception detection and defenses are inadequate . Existing methods do not integrate with agent decision-making .
Approach:	They propose a framework that integrates hybrid-reward learning with asymmetric penalties and experience summarization to distill failure patterns into transferable guidance.
Outcome:	The proposed framework reduces deception susceptibility by 53.8% while maintaining task performance, establishing an effective foundation for robust web agent deployment.

Reasoning Beyond Literal: Cross-style Multimodal Reasoning for Figurative Language Understanding (2026.findings-eacl)

Copied to clipboard

Seyyed Saeid Cheshmi, Hahnemann Ortiz, James Mooney, Dongyeop Kang

Sentiment Analysis

Challenge:	figurative language is essential for expressing intent, emotion, and perspective . figural language is often dependent on Styles Reasoning, causing incongruities between expressions .
Approach:	They propose a framework that induces reasoning capabilities to compact vision–language models . figurative language is essential in expressing intent, emotion, and perspective .
Outcome:	The proposed framework can interpret multimodal figurative language, provide transparent reasoning traces, and generalize across multiple figurativ styles.

Iterative Self-Correction for Text-Driven Person Re-Identification with Large Vision-Language Models (2026.findings-acl)

Copied to clipboard

Guijin Luo, Zequn Xie, Sihang Cai, Chuxin Wang, Zhou Zhao, Yixuan Tang

AlignMMBench: Evaluating Chinese Multimodal Alignment in Large Vision-Language Models (2025.acl-long)

Challenge:	Existing methods for Person Re-Identification (ReID) adopt a static "one-pass" paradigm, converting images to text once for retrieval.
Approach:	They propose a framework that reformulates ReID as an iterative "Think-and-Refine" process.
Outcome:	The proposed framework outperforms state-of-the-art methods in complex occlusion scenarios.

Copied to clipboard

Yuhang Wu, Wenmeng Yu, Yean Cheng, Yan Wang, Xiaohan Zhang, Jiazheng Xu, Ming Ding, Yuxiao Dong

Benchmarking Vision Language Models for Cultural Understanding (2024.emnlp-main)

Challenge:	Existing benchmarks focus on basic abilities using nonverbal methods, such as yes-no and multiple-choice questions.
Approach:	They propose a benchmark that provides more nuanced evaluations of alignment capabilities for large Vision-Language Models (VLMs) they use a rule-calibrated evaluator that exceeds GPT-4's evaluation ability and a “alignment score” to assess the robustness and stability of models across diverse prompts.
Outcome:	The proposed benchmark covers 13 tasks across three categories and includes both single-turn and multi-turn dialogue scenarios.

Copied to clipboard

Shravan Nayak, Kanishk Jain, Rabiul Awal, Siva Reddy, Sjoerd Steenkiste, Lisa Hendricks, Karolina Stanczak, Aishwarya Agrawal

ReasonRec: A Reasoning-Augmented Multimodal Agent for Unified Recommendation (2026.findings-acl)

Challenge:	Recent multimodal vision-language models have shown impressive performance in tasks such as image-to-text generation, visual question answering, and image captioning.
Approach:	They propose a visual question-answering benchmark to assess VLMs' cultural understanding of various facets of culture from 11 countries across 5 continents.
Outcome:	The visual question-answering benchmark aims to assess VLMs' cultural understanding across regions.

Copied to clipboard

Yihua Zhang, Mingfu Liang, Jiyan Yang, Rong Jin, Wen-Yen Chen, Yiping Han, Huayu Li, Buyun Zhang, Liang Luo, Luke Simon, Sijia Liu, Tianlong Chen, Xi Liu

Attacking Vision-Language Computer Agents via Pop-ups (2025.acl-long)

Challenge:	Recent advances in multimodal recommenders lack explicit reasoning and self-awareness of uncertainty.
Approach:	They propose a reasoning-augmented multimodal agent structured around a three-stage explicit reasoning pipeline.
Outcome:	The proposed agent improves ranking metrics and performance on four standard recommendation tasks across five real-world datasets.

Copied to clipboard

Yanzhe Zhang, Tao Yu, Diyi Yang

VLR-Bench: Multilingual Benchmark Dataset for Vision-Language Retrieval Augmented Generation (2025.coling-main)

Challenge:	Existing tools for analyzing and testing VLMs are lacking in understanding what types of attacks are possible and what types are effective.
Approach:	They propose to integrate pop-ups into existing agent testing environments to attack VLM agents by ignoring them.
Outcome:	The proposed attack success rate is 86% and decreases by 47% when integrating pop-ups into existing agent testing environments.

Copied to clipboard

Hyeonseok Lim, Dongjae Shin, Seohyun Song, Inho Won, Minjun Kim, Junghun Yuk, Haneol Jang, KyungTae Lim

Revisiting Classical Chinese Event Extraction with Ancient Literature Information (2025.acl-long)

Challenge:	Existing evaluation datasets for external knowledge-based VQA lack a capability to determine which passage is useful for answering queries.
Approach:	They propose a visual question answering benchmark for vision language models based on retrieval augmented generation (RAG) the proposed benchmark includes five input passages, a capability lacking in previous research.
Outcome:	The proposed benchmark includes five input passages and is validated using the state-of-the-art Llama3-based VLM, the Llava-Llamama-3 model.

Copied to clipboard

Xiaoyi Bao, Zhongqing Wang, Jinghang Gu, Chu-Ren Huang

Information Extraction & Text Mining

Challenge:	Existing studies on classical Chinese event extraction focus on grafting the complex modeling from English or modern Chinese works, neglecting the unique characteristic of this language.
Approach:	They propose a Literary Vision-Language Model (VLM) for classical Chinese event extraction . they integrate annotations, historical background and character glyphs to capture the inner- and outer-context information from the sequence.
Outcome:	The proposed model can capture the inner- and outer-context information at nearly zero cost.

GeoDANO: Geometric VLM with Domain Agnostic Vision Encoder (2025.findings-emnlp)

Copied to clipboard

Seunghyuk Cho, Zhenyue Qin, Yang Liu, Youngbin Choi, Seungbeom Lee, Dongwoo Kim

Does Acceleration Cause Hidden Instability in Vision Language Models? Uncovering Instance-Level Divergence Through a Large-Scale Empirical Study (2025.emnlp-main)

Challenge:	GeoDANO is a geometric vision-language model with a domain-agnostic vision encoder . it is currently limited to recognizing geometric features in general-purpose VLMs .
Approach:	They propose a geometric vision-language model with a domain-agnostic vision encoder for plane geometry problems.
Outcome:	The proposed model outperforms vision encoders in recognizing geometric features . it outperformed specialized methods for plane geometry problems and GPT-4o on MathVerse .

Copied to clipboard

Yizheng Sun, Hao Li, Chang Xu, Hongpeng Zhou, Chenghua Lin, Riza Batista-Navarro, Jingyuan Sun

VLM Is a Strong Reranker: Advancing Multimodal Retrieval-augmented Generation via Knowledge-enhanced Reranking and Noise-injected Training (2025.findings-emnlp)

Challenge:	Current acceleration evaluations focus on minimal overall performance degradation . however, accelerated models can exhibit significant changes in instance-level predictions .
Approach:	They investigate whether accelerated vision-Language Models can still give the same answers as before . they found that accelerated models changed original answers up to 20% of the time .
Outcome:	The results show that accelerated models changed their original answers up to 20% of the time.

Copied to clipboard

Zhanpeng Chen, Chengjin Xu, Yiyan Qi, Xuhui Jiang, Jian Guo

DocReRank: Single-Page Hard Negative Query Generation for Training Multi-Modal RAG Rerankers (2025.emnlp-main)

Challenge:	a significant drawback of Vision-language Models is their reliance on static training data, leading to outdated information and limited contextual awareness.
Approach:	They propose a framework with knowledge-enhanced reranking and noise-injected training to improve the VLM's ranking ability.
Outcome:	The proposed framework is based on a simple yet effective instruction template and is able to induce its ranking ability and serve it as a reranker to precisely filter the top-k retrieved images.

Copied to clipboard

Navve Wasserman, Oliver Heinimann, Yuval Golbari, Tal Zimbalist, Eli Schwartz, Michal Irani

MM-Reasoner: A Multi-Modal Knowledge-Aware Framework for Knowledge-Based Visual Question Answering (2023.findings-emnlp)

Challenge:	Existing models focus on identifying relevant documents, but embedding similarity often limits accuracy.
Approach:	They propose a method to generate hard negative queries per page instead of negative pages per query . they propose to refine ranking of an initial set of retrieved documents using hard negative mining .
Outcome:	The proposed approach outperforms existing models and significantly improves retrieval performance.

Copied to clipboard

Mahmoud Khademi, Ziyi Yang, Felipe Frujeri, Chenguang Zhu

MemeGuard: An LLM and VLM-based Framework for Advancing Content Moderation via Meme Intervention (2024.acl-long)

Challenge:	Recent knowledge-based visual question answering approaches miss visual information captured by captions and cannot fully utilize the visual information required to answer the question.
Approach:	They propose a framework that extracts visual information from an image and prompts an LLM to extract query-specific knowledge from the extracted textual information.
Outcome:	Empirical results show that MM-Reasoner achieves state-of-the-art performance on several KVQA datasets.

Copied to clipboard

Prince Jha, Raghav Jain, Konika Mandal, Aman Chadha, Sriparna Saha, Pushpak Bhattacharyya

Direct Metric Optimization for Image Captioning through Reward-Weighted Augmented Data Utilization (2024.acl-long)

Challenge:	Existing studies on content moderation of toxic memes focus on text-based content . current research neglects the widespread influence of multimodal content like memes .
Approach:	They propose a framework leveraging Large Language Models and Visual Language Model (VLMs) for meme intervention.
Outcome:	The proposed framework enables users to generate relevant and effective responses to toxic memes.

Copied to clipboard

Takumi Takada, Yuma Suzuki, Hiroki Takushima, Hayato Tanoue, Haruki Sato, Aiswariya Kumar, Hiroki Nishihara, Takayuki Hori, Kazuya Ueki

Value-Spectrum: Quantifying Preferences of Vision-Language Models via Value Decomposition in Social Media Contexts (2025.acl-long)

Challenge:	Recent large-scale vision language models (VLMs) lack continuity between learning objective and performance metrics.
Approach:	They propose a lightweight final-metric-optimizing training method that replaces the expensive exploration process in RL with an offline, diverse text data augmentation method.
Outcome:	The proposed method achieves comparable performance to state-of-the-art RL method while saving hundreds of times more model forwarding iterations and greater amounts of computation time.

Copied to clipboard

Jingxuan Li, Yuning Yang, Shengqi Yang, Linfan Zhang, Ying Nian Wu

Aligning VLM Assistants with Personalized Situated Cognition (2025.acl-long)

Challenge:	Recent advances in Vision-Language Models (VLMs) have broadened the scope of multimodal applications, but evaluations often neglect abstract dimensions such as personality traits and human values.
Approach:	They propose a Visual Question Answering (VQA) benchmark based on Schwartz’s value dimensions that capture core human values guiding people’s preferences and actions.
Outcome:	The proposed model can be used to evaluate visual question answering (VQA) tasks and to simulate diverse personas.

Copied to clipboard

Yongqi Li, Shen Zhou, Xiaohu Li, Xin Miao, Jintao Wen, Mayi Xu, Jianhao Chen, Birong Pan, Hankun Kang, Yuanyuan Zhu, Ming Zhong, Tieyun Qian

EchoVLM: Dynamic Mixture-of-Experts Vision-Language Model for Universal Ultrasound Intelligence (2026.acl-long)

Challenge:	Existing studies on vision-language models aligned with general human objectives have not been successful because people with diversified backgrounds have different cognition even in the same situation.
Approach:	They propose to characterize individuals based on the sociological concept of Role-Set and then evaluate their actions to see whether personalized alignment is achieved.
Outcome:	The proposed framework constructs a cognition-aware and action-based reward model for personalized alignment.

Copied to clipboard

Chaoyin She, Ruifang Lu, Lida Chen, Wei Wang, Qinghua Huang

Exploring How Generative MLLMs Perceive More Than CLIP with the Same Vision Encoder (2025.acl-long)

Challenge:	Ultrasound is the preferred early cancer screening modality due to non-ionizing radiation, cost-effectiveness, and real-time imaging.
Approach:	They propose to use ultrasound-tailored vision-language models with a mixture-of-experts architecture to train ultrasound-specific knowledge across seven anatomical systems.
Outcome:	The proposed model outperforms Qwen2-VL by 7.58 BLEU-1 and 3.45 ROUGE-1 points in report generation.

Copied to clipboard

Siting Li, Pang Wei Koh, Simon Shaolei Du

PII-VisBench: Evaluating Personally Identifiable Information Safety in Vision Language Models Along a Continuum of Visibility (2026.findings-acl)

Challenge:	Recent studies show that CLIP models struggle with visual reasoning tasks . despite the success of Contrastive Language-Image Pretraining, there are still limitations .
Approach:	They propose to use a visual encoder to train CLIP-like models for fine-grained visual reasoning tasks.
Outcome:	The proposed models outperform CLIP-like encoders in visual reasoning tasks . the study highlights the importance of VLM architectural choices .

Copied to clipboard

G M Shahariar, Zabir Al Nazi, Md Olid Hasan Bhuiyan, Zhouxing Shi

AutoGUI: Scaling GUI Grounding with Automatic Functionality Annotations from LLMs (2025.acl-long)

Challenge:	Existing evaluations of PII leakage ignore how a subject’s online presence affects privacy alignment.
Approach:	They propose a benchmark that evaluates safety through the continuum of online presence by stratifying 200 subjects into four visibility categories: high, medium, low, and zero.
Outcome:	The proposed model stratifies 200 subjects into four visibility categories based on the extent and nature of their information available online.

Copied to clipboard

Hongxin Li, Jingfan Chen, Jingran Su, Yuntao Chen, Li Qing, Zhaoxiang Zhang

Fine-Grained Image-Text Alignment in Medical Imaging Enables Explainable Cyclic Image-Report Generation (2024.acl-long)

Challenge:	Existing datasets for UI-VLMs contain large-scale context-free element annotations or contextualized functional descriptions for elements at a small scale.
Approach:	They propose an auto-annotation pipeline that generates massive UI element functionality annotations based on UI content changes induced by interacting with the elements.
Outcome:	The proposed pipeline improves accuracy and scales well with human evaluation of a high-quality AutoGUI-704k dataset.

Copied to clipboard

Wenting Chen, Linlin Shen, Jingyang Lin, Jiebo Luo, Xiang Li, Yixuan Yuan

What Users Leave Unsaid: Under-Specified Queries Limit Vision-Language Models (2026.findings-acl)

Challenge:	Fine-grained vision-language models (VLMs) have been widely used for inter-modality local alignment between fixed patches and textual words, but they provide incomplete representations of lesions.
Approach:	They propose an Adaptive patch-word Matching model to correlate chest X-ray (CXR) image regions with words in medical reports and apply it to CXR-report generation to provide explicit explanations.
Outcome:	The proposed model correlates chest X-ray image regions with words in medical reports and provides explanations for the generation process.

Copied to clipboard

Dasol Choi, Guijin Son, Hanwool Lee, Minhyuk Kim, Hyunwoo Ko, Teabin Lim, Eungyeol Ahn, Jungwhan Kim, Seunghyeok Hong, Youngsook Song

MediVLM: A Vision Language Model for Radiology Report Generation from Medical Images (2025.findings-emnlp)

Challenge:	HAERAE-Vision benchmarks feature clear, explicit prompts but are often informal and underspecified . state-of-the-art models achieve under 50% on original queries, compared to GPT-5 and Gemini 2.5 Pro .
Approach:	They propose a benchmark of 653 real-world visual questions from Korean online communities . they find that even state-of-the-art models achieve under 50% on original queries .
Outcome:	HAERAE-Vision benchmarks from Korean online communities yield 1,306 query variants . state-of-the-art models achieve under 50% on original queries, compared with smaller models . authors show that query explicitation alone yields 8 to 22 point improvements .

Copied to clipboard

Debanjan Goswami, Ronast Subedi, Shayok Chakraborty

If CLIP Could Talk: Understanding Vision-Language Model Representations Through Their Preferred Concept Descriptions (2024.emnlp-main)

Challenge:	Existing methods for radiology report generation from medical images are incomplete and inconsistent, fail to focus on informative regions within an image and impose strong annotation assumptions for model training.
Approach:	They propose a vision language model (VLM) for radiology report generation from medical images that uses a pre-trained object detector to extract the salient anatomical regions from images, an image encoder, a text encoder and a transformer based decoder to generate the final report.
Outcome:	The proposed model generates radiology reports even when no reports are available for training.

Copied to clipboard

Reza Esfandiarpoor, Cristina Menghini, Stephen Bach

What Do VLMs NOTICE? A Mechanistic Interpretability Pipeline for Gaussian-Noise-free Text-Image Corruption and Evaluation (2025.naacl-long)

Challenge:	Recent studies assume that VLMs prioritize visual attributes to represent concepts.
Approach:	They propose a novel approach to characterize features important for VLMs using reinforcement learning.
Outcome:	The proposed approach characterizes features that are important for VLMs . it shows that spurious descriptions have a major role in VLM representations despite providing no helpful information.

Copied to clipboard

Michal Golovanevsky, William Rudman, Vedant Palit, Carsten Eickhoff, Ritambhara Singh

Investigating Cross-Modal Skill Injection: Scenarios, Methods, and Hyperparameters (2026.acl-long)

Challenge:	Vision-Language Models (VLMs) have gained prominence due to their success in solving complex cross-modal tasks.
Approach:	They propose a Gaussian-Noise-free pipeline for mechanistic interpretability in VLMs that introduces Semantic Image Pairs corruption, the first visual counterpart to Symmetric Token Replacement for text.
Outcome:	The proposed pipeline identifies a set of “universal attention heads” in BLIP and LLaVA that consistently contribute across different tasks and modalities.

Copied to clipboard

Zhiyu Xu, Lean Wang, Yuanxin Liu, Lei Li, Hao Zhou, Fandong Meng, Jie Zhou, Xu Sun

SilVar: Speech-Driven Multimodal Model for Reasoning Visual Question Answering and Object Localization (2025.emnlp-main)

Challenge:	Existing research lacks systematic analysis of the applicability and methodology of cross-modal skill injection.
Approach:	They investigate the applicability and methodology of cross-modal skill injection by integrating a domain-expert LLM into a VLM.
Outcome:	The proposed method enables transfer of domain-specific expertise from Large Language Models (LLMs) to VLMs without incurring additional training data requirements or significant computational overhead.

Copied to clipboard

Tan-Hanh Pham, Le Hoang Nam, Phu-Vinh Nguyen, Chris Ngo, Truong-Son Hy

Fair Federated Learning with Biased Vision-Language Models (2024.findings-acl)

Challenge:	Visual Language Models have demonstrated remarkable capabilities across various tasks, including visual question answering and image captioning.
Approach:	They propose an end-to-end multimodal model that leverages speech instructions for reasoning-based visual question answering.
Outcome:	The proposed model can process and explain visual scenes from spoken input, moving beyond simple object recognition to reasoning-based interactions.

Copied to clipboard

Huimin Zeng, Zhenrui Yue, Yang Zhang, Lanyu Shang, Dong Wang

MobileVLM: A Vision-Language Model for Better Intra- and Inter-UI Understanding (2024.findings-emnlp)

Challenge:	Existing literature ignores the inherent group unfairness within CLIP and its ethical implications on FL applications.
Approach:	They propose a fairness-aware adaptation framework for CLIP in federated learning . they propose to leverage biased pre-trained VLMs to build fair FL frameworks .
Outcome:	The proposed framework addresses unique bias in FL, triggered by data heterogeneity . it trains a fair FL model with fairness-aware deep visual prompting (DVP) Extensive results on human face attribute recognition (FAR) applications show it outperforms state-of-the-art FL models .

Copied to clipboard

Qinzhuo Wu, Weikai Xu, Wei Liu, Tao Tan, Liujian Liujianfeng, Ang Li, Jian Luan, Bin Wang, Shuo Shang

Large Language Models Know What is Key Visual Entity: An LLM-assisted Multimodal Retrieval for VQA (2024.emnlp-main)

Challenge:	Recent mobile AI agents based on VLMs lack basic mobile capabilities due to their pre-trained nature.
Approach:	They propose a mobile AI agent based on VLMs that includes additional pre-training stages to enhance both intra- and inter-UI understanding.
Outcome:	The proposed model outperforms existing VLMs on the Chinese mobile dataset Mobile3M .

Copied to clipboard

Pu Jian, Donglei Yu, Jiajun Zhang

Beyond Screenshots: Evaluating VLMs’ Understanding of UI Animations (2026.findings-acl)

Challenge:	Existing visual language models struggle to capture longtail knowledge in the real world due to redundant visual information.
Approach:	They propose a method leveraging the reasoning capability of a large language model to identify key visual entities.
Outcome:	The proposed method outperforms other strong visual language model-based systems in two knowledge-intensive VQA benchmarks and performs comparably to models with 1-2 orders larger parameters.

Copied to clipboard

Chen Liang, Xirui Jiang, Naihao Deng, Eytan Adar, Anhong Guo

Challenge:	Recent studies of Vision Language Models (VLMs) for UI understanding have focused primarily on static screenshots, leaving it unclear how well these models handle dynamic UI animations.
Approach:	They evaluate UI animation models' ability to perceive animation effects and interpret animation meaning . they use motion, context, and perceptual cues to probe factors affecting VLM performance .
Outcome:	The proposed model can detect primitive motion, but its interpretation is inconsistent . the proposed model is based on 300 annotated UI animation videos .

ProcWorld: Benchmarking Large Model Planning in Reachability-Constrained Environments (2025.emnlp-main)

Copied to clipboard

Dong Wang, Xinghang Li, Zhengshen Zhang, Jirong Liu, Xiao Ma, Hanbo Zhang, Tao Kong, Huaping Liu

MapNav: A Novel Memory Representation via Annotated Semantic Maps for VLM-based Vision-and-Language Navigation (2025.acl-long)

Challenge:	Existing benchmarks for embodied spatial reasoning and long-term planning are non-trivial due to the combinatorial complexity of long-horizon abstract reasoning.
Approach:	They propose a large-scale benchmark for partially observable embodied spatial reasoning and long-term planning with large language models and vision language models.
Outcome:	The proposed model performs better in 16 task types, 5,000 rooms, and over 10 million evaluation trajectories with diverse data distribution.

Copied to clipboard

Lingfeng Zhang, Xiaoshuai Hao, Qinwen Xu, Qiang Zhang, Xinyao Zhang, Pengwei Wang, Jing Zhang, Zhongyuan Wang, Shanghang Zhang, Renjing Xu

ViCor: Bridging Visual Understanding and Commonsense Reasoning with Large Language Models (2024.findings-acl)

Challenge:	Vision-language navigation (VLN) is a key task in Embodied AI . traditional approaches rely on historical observations as spatio-temporal contexts for decision making .
Approach:	They propose a vision-language navigation model that leverages an annotation system to replace historical frames.
Outcome:	The proposed model can be used as a new memory representation method in vision-language navigation . it can be applied to simulated and real-world environments, and it is validated by experiments .

Copied to clipboard

Kaiwen Zhou, Kwonjoon Lee, Teruhisa Misu, Xin Wang

Reasoning

Challenge:	Existing methods for visual commonsense reasoning (VCR) use pre-trained large language models and pre-training visionlanguage models.
Approach:	They propose a collaborative approach where pre-trained LLMs serve as problem classifiers to analyze problem category and either use VLMs to answer directly or actively instruct LLM to gather relevant visual elements to support potential commonsense inferences.
Outcome:	The proposed approach outperforms all other methods without in-domain fine-tuning on two VCR benchmark datasets.

Repairs in a Block World: A New Benchmark for Handling User Corrections with Multi-Modal Language Models (2024.emnlp-main)

Copied to clipboard

Javier Chiyah-Garcia, Alessandro Suglia, Arash Eshghi

Cultivating Gaming Sense for Yourself: Making VLMs Gaming Experts (2025.acl-long)

Challenge:	In dialogue, the addressee may misunderstand the speaker and respond erroneously.
Approach:	They collect, analyse, and publicly release a dataset of multi-modal TPR sequences in dialogue . they evaluate several state-of-the-art Vision and Language Models across multiple settings .
Outcome:	The proposed model underperforms in a human-robot interaction task compared to humans . the proposed model can benefit from specialised losses targeting relevant tokens .

Copied to clipboard

Wenxuan Lu, Jiangyang He, Zhanqiu Zhang, Steven Y. Guo, Tianning Zang

AlanaVLM: A Multimodal Embodied AI Foundation Model for Egocentric Video Understanding (2024.findings-emnlp)

Challenge:	Recent efforts leverage Vision Language Models (VLMs) as direct controllers, often pausing the game to analyze screens and plan action through language reasoning.
Approach:	They propose a paradigm shift in gameplay agent design that uses Vision Language Models as a developer instead of direct control.
Outcome:	The proposed framework achieves fluent gameplay in diverse genres, including ACT, FPS, and Flappy Bird, setting a new benchmark for game-playing agents.

Copied to clipboard

Alessandro Suglia, Claudio Greco, Katie Baker, Jose Part, Ioannis Papaioannou, Arash Eshghi, Ioannis Konstas, Oliver Lemon

Language-Informed Synthesis of Rational Agent Models for Grounded Theory-of-Mind Reasoning On-the-fly (2025.findings-emnlp)

Challenge:	Current Vision-Language Models (VLMs) focus on third-person view videos, neglecting the richness of egocentric perceptual experience.
Approach:	They propose to use the Egocentric Video Understanding Dataset (EVUD) to train VLMs on video captioning and question answering tasks specific to egocentric videos.
Outcome:	The proposed model outperforms open-source models including strong Socratic models using GPT-4 as a planner by 3.6% and outperformed Claude 3 and Gemini Pro Vision 1.0.

Copied to clipboard

Lance Ying, Ryan Truong, Katherine M. Collins, Cedegao E. Zhang, Megan Wei, Tyler BrookeWilson, Tan Zhi-Xuan, Lionel Wong, Joshua B. Tenenbaum

MASSV: Multimodal Adaptation and Self-Data Distillation for Speculative Decoding of Vision-Language Models (2025.findings-emnlp)

Challenge:	Language is a powerful source of information in social settings, especially in novel situations where language can provide both abstract information about the environment dynamics and concrete specifics about an agent that cannot be easily visually observed.
Approach:	They propose a language-informed rational agent synthesis framework that integrates linguistic and visual inputs to draw context-specific social inferences.
Outcome:	The proposed framework outperforms ablations and state-of-the-art models on a range of social reasoning tasks derived from cognitive science experiments.

Copied to clipboard

Mugilan Ganesan, Shane Segal, Ankur Aggarwal, Nish Sinnadurai, Sean Lie, Vithursan Thangarasa

Prometheus-Vision: Vision-Language Model as a Judge for Fine-Grained Evaluation (2024.findings-acl)

Challenge:	Speculative decoding of vision-language models provides a novel way to accelerate language model inference by enabling a lightweight draft model to propose multiple tokens that a larger target model verifies simultaneously.
Approach:	They propose a technique that allows a lightweight draft model to propose multiple tokens that a larger target model verifies simultaneously.
Outcome:	The proposed technique increases accepted length by 30% and delivers speedups of up to 1.46x compared to conventional text-only drafting baselines on visually-grounded tasks.

Copied to clipboard

Seongyun Lee, Seungone Kim, Sue Park, Geewook Kim, Minjoon Seo

Believing without Seeing: Quality Scores for Contextualizing Vision-Language Model Explanations (2026.acl-long)

Challenge:	Existing metrics for long-form text outputs are prone to biases and scaling up is expensive.
Approach:	They propose to evaluate VLMs with VLM feedback dataset . they use 15K customized score rubrics to train Prometheus-Vision .
Outcome:	The proposed model shows highest correlation with human evaluators and GPT-4V among open-source models.

Copied to clipboard

Keyu He, Tejas Srinivasan, Brihi Joshi, Xiang Ren, Jesse Thomason, Swabha Swayamdipta

Explainability & Interpretability in NLP

Challenge:	Prior work has found that explanations can easily convince users that inaccurate VLM predictions are correct.
Approach:	They propose to evaluate two complementary qualities of VLM-generated explanations via two quality scoring functions to improve their accuracy.
Outcome:	The proposed explanations improve accuracy on the A-OKVQA, VizWiz, and MMMU-Pro tasks by 11.1%, including a 15.4% reduction in falsely believing incorrect predictions.

MoLoRAG: Bootstrapping Document Understanding via Multi-modal Logic-aware Retrieval (2025.emnlp-main)

Copied to clipboard

Xixi Wu, Yanchao Tan, Nan Hou, Ruiyang Zhang, Hong Cheng

VIGNETTE: Socially Grounded Bias Evaluation for Vision-Language Models (2026.acl-long)

Challenge:	Document Understanding is a foundational AI capability with broad applications . Large Vision-Language Models (LLMs) can't handle multi-page document comprehension . a logic-aware retrieval framework for multi-modal, multi- page document understanding is proposed .
Approach:	They propose a logic-aware retrieval framework for multi-modal, multi-page document understanding . MoLoRAG uses semantic and logical relevance to deliver more accurate retrieval .
Outcome:	The proposed framework improves on four DocQA datasets and demonstrates 9.68% accuracy improvement over existing methods.

Copied to clipboard

Chahat Raj, Bowen Wei, Aylin Caliskan, Antonios Anastasopoulos, Ziwei Zhu

Cross-Modal Taxonomic Generalization in (Vision-) Language Models (2026.acl-long)

Challenge:	Existing studies on VLM bias focus on portrait-style images and gender-occupation associations . existing studies ignore broader and more complex social stereotypes and their implied harm .
Approach:	They propose a large-scale VQA benchmark for evaluating bias in vision-language models . they use a question-answering framework that spans factuality, perception, stereotyping, and decision making .
Outcome:	The proposed framework examines bias in vision-language models using 30M+ images . findings reveal subtle, multifaceted, and surprising stereotypical patterns .

Copied to clipboard

Tianyang Xu, Marcelo Sandoval-Castañeda, Karen Livescu, Greg Shakhnarovich, Kanishka Misra

IdealGPT: Iteratively Decomposing Vision and Language Reasoning via Large Language Models (2023.findings-emnlp)

Challenge:	Existing studies have shown that language models learn from surface form to learn from more grounded evidence.
Approach:	They propose to use a vision-language model to learn hypernyms from images . they find that the model can recover this knowledge and generalize even when there is no hypernomia in the image.
Outcome:	The proposed model can recover this knowledge and generalize even when the model receives no evidence of hypernyms during training.

Copied to clipboard

Haoxuan You, Rui Sun, Zhecan Wang, Long Chen, Gengyu Wang, Hammad Ayyubi, Kai-Wei Chang, Shih-Fu Chang

Selective “Selective Prediction”: Reducing Unnecessary Abstention in Vision-Language Reasoning (2024.findings-acl)

Challenge:	Existing approaches to decompose VL reasoning rely on domain-specific sub-question decomposing models.
Approach:	They propose a framework that iteratively decomposes VL reasoning using large language models.
Outcome:	The proposed framework outperforms existing models on multiple VL reasoning tasks.

Copied to clipboard

Tejas Srinivasan, Jack Hessel, Tanmay Gupta, Bill Yuchen Lin, Yejin Choi, Jesse Thomason, Khyathi Chandu

Collaborative Multi-Agent Scripts Generation for Enhancing Imperfect-Information Reasoning in Murder Mystery Games (2026.findings-acl)

Challenge:	ReCoVERR reduces the over-abstention of a vision-language system with low tolerance for inaccurate predictions without increasing the error rate of the system’s predictions.
Approach:	They propose an inference-time algorithm to reduce the over-abstention of a selective vision-language system without increasing the error rate of the system’s predictions.
Outcome:	ReCoVERR reduces the over-abstention of a vision-language system without increasing the error rate of the system’s predictions.

Copied to clipboard

Keyang Zhong, Junlin Xie, Hefeng Wu, Haofeng Li, Guanbin Li

VLMInferSlow: Evaluating the Efficiency Robustness of Large Vision-Language Models as a Service (2025.acl-long)

Challenge:	Vision-language models have shown impressive capabilities in perceptual tasks . however, they degrade in complex multi-hop reasoning under multi-player game settings .
Approach:	They propose a multi-agent framework for evaluating and synthesizing role-driven game scripts . they use curated and synthetic datasets to model uncertainty and deception .
Outcome:	The proposed model significantly boosts the performance of VLMs in narrative reasoning and hidden fact extraction under uncertain, adversarial, and socially complex conditions.

Copied to clipboard

Xiasi Wang, Tianliang Yao, Simin Chen, Runqi Wang, Lei Ye, Kuofeng Gao, Yi Huang, Yuan Yao

Why do LLaVA Vision-Language Models Reply to Images in English? (2024.findings-emnlp)

Challenge:	Existing studies evaluate efficiency robustness of vision-language models under unrealistic assumptions, requiring access to model architecture and parameters.
Approach:	They propose a novel approach to evaluate VLM efficiency robustness in a realistic black-box setting.
Outcome:	The proposed approach generates adversarial images with imperceptible perturbations, increasing the computational cost by up to 128.47%.

Copied to clipboard

Musashi Hinck, Carolin Holtermann, Matthew Olson, Florian Schneider, Sungduk Yu, Anahita Bhiwandiwalla, Anne Lauscher, Shao-Yen Tseng, Vasudev Lal

Targeted Exploration via Unified Entropy Control for Reinforcement Learning (2026.findings-acl)

Challenge:	Including an image in a multimodal query significantly increases the likelihood of the model returning an English response regardless of the language of the query.
Approach:	They propose a two-pronged approach that combines extensive ablation of the design space with a mechanistic analysis of the models’ internal representations of image and text inputs.
Outcome:	The proposed approach reduces the multilingual error by switching the language backbone for a bilingual language model.

Copied to clipboard

Chen Wang, Lai Wei, Yanzhi Zhang, Chenyang Shao, Zedong Dan, Weiran Huang, Ge Lan, Yue Wang

Argus: Benchmarking and Enhancing Vision-Language Models for 3D Radiology Report Generation (2025.findings-acl)

Challenge:	Existing methods for group relative policy optimization suffer from entropy collapse . Existing exploration methods introduce additional bias or variance during exploration, making it difficult to maintain stability.
Approach:	They propose a framework that provides targeted mechanisms for exploration and stabilization.
Outcome:	The proposed framework expands search space on difficult prompts while preventing entropy growth uncontrollably.

Copied to clipboard

Che Liu, Zhongwei Wan, Yuqi Wang, Hui Shen, Haozhe Wang, Kangyu Zheng, Mi Zhang, Rossella Arcucci

Can VLMs Recall Factual Associations From Visual References? (2025.findings-emnlp)

Challenge:	Existing work on 3D radiograph report generation focuses on 2D images, but 3D medical images provide more comprehensive diagnostic information.
Approach:	They propose a comprehensive training recipe for building high-performing VLMs for 3DRRG using a publicly available 3D CT-report dataset.
Outcome:	The proposed model achieves superior performance across different model sizes and input 3D medical image resolutions.

Copied to clipboard

Dhananjay Ashok, Ashutosh Chaubey, Hirona Jacqueline Arai, Jonathan May, Jesse Thomason

EfficientVLM: Fast and Accurate Vision-Language Models via Knowledge Distillation and Modal-adaptive Pruning (2023.findings-acl)

Challenge:	a systematic deficiency in the multimodal grounding of Vision Language Models is identified . VLMs can recall factual associations when provided a textual reference to an entity .
Approach:	They identify a systematic deficiency in the multimodal grounding of Vision Language Models . they show that VLMs struggle to link their internal knowledge of an entity with its image representation .
Outcome:	The study shows that VLMs struggle to link internal knowledge with image representations . the findings provide recommendations for future research .

Copied to clipboard

Tiannan Wang, Wangchunshu Zhou, Yan Zeng, Xinsong Zhang

VisualEDU: A Benchmark for Assessing Coding and Visual Comprehension through Educational Problem-Solving Video Generation (2025.findings-emnlp)

Challenge:	Pre-trained vision-language models have achieved impressive results in a range of vision-linguistic tasks.
Approach:	They propose a distilling then pruning framework to compress large vision-language models into smaller, faster ones.
Outcome:	The proposed framework reduces the size of a pre-trained large vision-language model and improves its performance on vision-linguistic tasks.

Copied to clipboard

Hao Chen, Tianyu Shi, Pengran Huang, Zeyuan Li, Jiahui Pan, Qianglong Chen, Lewei He

Retrieval-augmented GUI Agents with Generative Guidelines (2025.emnlp-main)

Challenge:	VisualEDU is a benchmark to evaluate VLMs' ability to produce coherent video from text . advanced proprietary models show promise, but struggle with increasing task complexity .
Approach:	VisualEDU is a benchmark to evaluate VLMs' ability to produce coherent video from text . it integrates meta-prompt learning, visual and code feedback, and a drawing toolkit to enhance output quality.
Outcome:	VisualEDU is a benchmark to evaluate VLMs' ability to produce coherent video from text . it integrates meta-prompt learning, visual and code feedback, and a drawing toolkit to improve output quality.

Copied to clipboard

Ran Xu, Kaixin Ma, Wenhao Yu, Hongming Zhang, Joyce C. Ho, Carl Yang, Dong Yu

Information Retrieval

Challenge:	GUI agents powered by vision-language models struggle with real-world tasks due to their complex nature and limited training data.
Approach:	They propose a lightweight vision-language model that leverages web tutorials at inferencetime to synthesize GUI agents.
Outcome:	The proposed agent outperforms baseline GUI agents and surpasses other inference baselines by 2.6% to 13.3% across two model sizes.

Vision-Flan: Scaling Human-Labeled Tasks in Visual Instruction Tuning (2024.findings-acl)

Copied to clipboard

Zhiyang Xu, Chao Feng, Rulin Shao, Trevor Ashby, Ying Shen, Di Jin, Yu Cheng, Qifan Wang, Lifu Huang

JARVIS-VLA: Post-Training Large-Scale Vision Language Models to Play Visual Games with Keyboards and Mouse (2025.findings-acl)

Challenge:	Recent vision-language models (VLMs) have shown impressive capabilities as general visual assistants, but there are two challenges to their performance: (1) lacking task diversity in pretraining and visual instruction tuning; (2) annotation error and bias in GPT-4 synthesized instruction tuning data.
Approach:	They propose a two-stage instruction tuning framework that fine tunes VLMs firstly and further tuned on GPT-4 synthesized data.
Outcome:	The proposed framework outperforms the traditional single-stage visual instruction tuning framework and achieves state-of-the-art performance across a wide range of multi-modal evaluation benchmarks.

Copied to clipboard

Muyao Li, Zihao Wang, Kaichen He, Xiaojian Ma, Yitao Liang

Reasoning Paths with Reference Objects Elicit Quantitative Spatial Reasoning in Large Vision-Language Models (2024.emnlp-main)

Challenge:	Visual Language Action models have shown promise in decision-making tasks, but have been neglected in previous work .
Approach:	They propose a new paradigm for visual language action models that enhances the foundation model prior to action-specific tuning by first post-training it on a curated set of visual and linguistic tasks using self-supervised learning.
Outcome:	The proposed model outperforms the best agent baseline on a diverse set of atomic tasks and surpasses imitation learning-based policies in Minecraft.

Copied to clipboard

Yuan-Hong Liao, Rafid Mahmood, Sanja Fidler, David Acuna

Vision-Language Models Struggle to Align Entities across Modalities (2025.findings-acl)

Challenge:	Despite recent advances in visual language models, their ability to quantitatively reason about object sizes and distances remains underexplored.
Approach:	They propose a manually annotated benchmark of 241 questions designed for quantitative spatial reasoning and a zero-shot prompting technique that encourages VLMs to use reference objects as visual cues.
Outcome:	The proposed technique improves the performance of the top-performing VLMs by 19 points when a reasoning path using a reference object emerges naturally in the response.

Copied to clipboard

Iñigo Alonso, Gorka Azkune, Ander Salaberria, Jeremy Barnes, Oier Lopez De Lacalle

e5-omni: Explicit Cross-modal Alignment for Omni-modal Embeddings (2026.findings-acl)

Challenge:	Several real-world applications require the ability to perform cross-modal entity linking . cross-functional entity linking is a skill needed for multimodal code generation and scene understanding .
Approach:	They propose a task and benchmark to evaluate cross-modal entity linking performance . they use visual scenes aligned with their textual representations to evaluate performance a question-answering task .
Outcome:	The proposed task and benchmark aims to improve cross-modal entity linking performance . it evaluates state-of-the-art vision-language models and humans on the task .

Copied to clipboard

Haonan Chen, Sicheng Gao, Radu Timofte, Tetsuya Sakai, Zhicheng Dou

V-ALPHASOCIAL: Benchmark and Self-Reflective Chain-of-Thought Generation for Visual Social Commonsense Reasoning (2025.findings-acl)

Challenge:	Recent omni-modal embeddings rely heavily on implicit alignment from pretrained visionlanguage models.
Approach:	They propose a lightweight explicit alignment recipe that adapts off-the-shelf VLMs into robust omni-modal embedding models.
Outcome:	The proposed model improves on MMEB-V2 and AudioCaps with a lightweight explicit alignment recipe.

Copied to clipboard

Zongyu Lin, Zhikun Xu, Xiaohan Song, Yixin Wan, Xingcheng Yao, Tsung-Han Lin, Selina Song, Pranav Subbaraman, Ben Zhou, Kai-Wei Chang, Yizhou Sun

Reasoning

Challenge:	Social commonsense reasoning is a multimodal task that requires both textual and visual cues.
Approach:	They propose a method that integrates visual cues into social commonsense reasoning tasks.
Outcome:	The proposed method improves social commonsense reasoning on a multimodal foundation model.

GaLa: Hypergraph-Guided Visual Language Models for Procedural Planning (2026.findings-acl)

Copied to clipboard

Kun Wang, Li Yiming, Mingcheng Qu, Aqiang Zhang, Guang Yang, Tonghua Su

Mixture-of-Prompt-Experts for Multi-modal Semantic Understanding (2024.lrec-main)

Challenge:	Existing methods for procedural planning over-rely on visual inputs and lack structured semantic information.
Approach:	They propose a vision–language framework for multimodal procedural planning that exploits implicit spatial relations and deep semantics encoded in object attributes.
Outcome:	The proposed framework outperforms existing methods in terms of execution success rate, LCS, and planning correctness.

Copied to clipboard

Zichen Wu, Hsiu-Yuan Huang, Fanyi Qu, Yunfang Wu

Treble Counterfactual VLMs: A Causal Approach to Hallucination (2025.findings-emnlp)

Challenge:	Multimodal semantic understanding is crucial for developing machines capable of interpreting complex interplay of text and visual information.
Approach:	They propose a multi-modal soft prompt framework that integrates three experts of soft prompts . they propose sarcasm detection and sentiment analysis tasks that are critical for few-shot learning .
Outcome:	The proposed model outperforms the 8.2B model InstructBLIP with 2% parameters . it significantly outperformed other prompt methods on VLMs or task-specific methods .

Copied to clipboard

Li Li, Jiashu Qu, Linxin Song, Yuxiao Zhou, Yuehan Qin, Tiankai Yang, Yue Zhao

See2Refine: Vision-Language Feedback Improves LLM-Based eHMI Action Designers (2026.acl-long)

Challenge:	Existing studies link hallucination to data or representation biases, but their causal origins remain unclear.
Approach:	They propose a causal framework to analyze and mitigate hallucination in vision-language models by using counterfactual analysis to estimate the Natural Direct Effect (NDE) of each modality and their interaction.
Outcome:	The proposed framework significantly reduces hallucination while preserving task performance while retaining reliability.

Copied to clipboard

Ding Xia, Xinyue Gui, Mark Colley, Fan Gao, Zhongyi Zhou, Dongyuan Li, Renhe Jiang, Takeo Igarashi

INDOTABVQA: A Benchmark for Cross-Lingual Table Understanding in Bahasa Indonesia Documents (2026.findings-acl)

Challenge:	External Human-Machine Interfaces (eHMIs) are emerging as promising solutions to address this communication gap.
Approach:	They propose a framework that uses vision-language models (VLMs) for perceptual evaluation as automated visual feedback to improve an LLM-based eHMI action designer.
Outcome:	The proposed framework outperforms prompt-only LLM designers and manually specified baselines in three eHMI modalities and multiple LLM model sizes.

Copied to clipboard

Somraj Gautam, Anathapindika Dravichi, Gaurav Harit

CharTide: Data-Centric Chart-to-Code Generation via Tri-Perspective Tuning and Inquiry-Driven Evolution (2026.acl-long)

Challenge:	INDOTABVQA provides a benchmark for evaluating cross-lingual Table Visual Question Answering (VQA) on real-world document images in Bahasa Indonesia.
Approach:	They propose a benchmark for evaluating cross-lingual Table Visual Question Answering on real-world document images in Bahasa Indonesia.
Outcome:	The proposed model improves on a 3B model and a LoRA- finetuned 7B model on Bahasa Indonesian document images by 11.6% and 17.8% respectively.

Copied to clipboard

Xiangxi Zheng, Kuang He, Jiayi Hu, Ping Yu, Rui Yan, Yuan Yao, Peng Hou, Anxiang Zeng, Alex Jinpeng Wang

VLN-NF: Feasibility-Aware Vision-and-Language Navigation with False-Premise Instructions (2026.acl-long)

Challenge:	Existing approaches to chart-to-code generation are constrained by data-centric limitations . authors present a new framework that redesigns both training and alignment data .
Approach:	They propose a data-centric framework that redesigns both training and alignment data for chart-to-code generation.
Outcome:	The proposed framework outperforms open-source baselines and is competitive with GPT-5.

Copied to clipboard

Hung-Ting Su, Ting-Jun Wang, Jia-Fong Yeh, Min Sun, Winston H. Hsu

Retrieval-enriched zero-shot image classification in low-resource domains (2024.emnlp-main)

Challenge:	Existing Vision-and-Language Navigation benchmarks assume instructions are feasible and the referenced target exists.
Approach:	They propose a benchmark with false-premise instructions where the target is absent . they propose supervised room-level navigation with LLM/VLM-driven in-room exploration .
Outcome:	The proposed benchmark produces false-premise goals that are plausible but factually incorrect . ROAM achieves the best REV-SPL among compared methods, while baselines often under-explore and terminate prematurely under unreliable instructions.

Copied to clipboard

Nicola Dall’Asen, Yiming Wang, Enrico Fini, Elisa Ricci

Information Retrieval

Challenge:	Low-resource domains are those where data or annotations are scarce.
Approach:	They propose a retrieval-based method for low-resource domains that trains without training . they use web-crawled databases to retrieve relevant textual information from query images .
Outcome:	The proposed method outperforms existing training-based methods in low-resource domains . it retrieves relevant textual information from large web-crawled databases .

ChemVLR: Prioritizing Reasoning in Perception for Chemical Vision-Language Understanding (2026.findings-acl)

Copied to clipboard

Xuanle Zhao, Xinyuan Cai, Xiang Cheng, Xiuyi Chen, Bo XU

Preserving Language Capabilities in Vision-Language Models via Representation Regulation (2026.findings-acl)

Challenge:	Currently, vision-Language Models are optimized for direct visual question-answering tasks.
Approach:	They propose a visual-language-based VLM that prioritizes reasoning within the perception process.
Outcome:	The proposed model outperforms existing models and domain-specific open-source models in the chemical domain.

Copied to clipboard

ZiXuan Chen, Juncheng Tao, Ziqian Zeng

Revisiting 3D LLM Benchmarks: Are We Really Testing 3D Capabilities? (2025.findings-acl)

Challenge:	Vision-Language Models (VLMs) provide a unified framework to process both text-only and vision-language tasks.
Approach:	They propose a method to reduce the distance between visual and textual representations by introducing a Representation Distribution Difference (RDD) loss.
Outcome:	Empirical evidence shows that finetuning VLMs on vision-language data has degraded language capabilities.

Copied to clipboard

Jiahe Jin, Yanheng He, Mingyan Yang

Challenge:	Current 3D LLMs are evaluated on Q&A or captioning tasks rather than specific downstream tasks like object detection.
Approach:	They propose principles for better assessing genuine 3D understanding by explicitly separating 3D abilities from 1D or 2D aspects when evaluating 3D LLMs.
Outcome:	The proposed methods are based on the “2D-Cheating” problem in 3D LLM evaluation, suggesting that they are ineffective .

AIM-CoT: Active Information-driven Multimodal Chain-of-Thought for Vision-Language Reasoning (2026.acl-long)

Copied to clipboard

Xiping Li, Jianghong Ma

Perceptual Hallucination in Vision–Language Models: Definition, Analysis and Verification (2026.findings-acl)

Challenge:	Existing methods for I-MCoT fail to capture dynamic needs of vision-language models . existing methods rely on attention signals, which are unreliable under severe granularity imbalance between brief textual query and informative image.
Approach:	They propose a framework that integrates specially selected visual evidence into the context of Vision-Language Models (VLMs) they propose 'AIM-CoT' to improve evidence selection and insertion triggering .
Outcome:	Experiments across three benchmarks and four backbones demonstrate the proposed framework’s consistent superiority.

Copied to clipboard

Taewook Hwang, Inbum Heo, Sung Jun Lee, Sangkeun Jung

Response Wide Shut? Surprising Observations in Basic Vision Language Model Capabilities (2025.acl-long)

Challenge:	Recent advances in large language models (LLMs) have dramatically improved text understanding and generation capabilities.
Approach:	They define perceptual hallucination as the phenomenon where VLMs generate information as if perceived, despite absent or damaged visual evidence.
Outcome:	The proposed model reduces hallucination exposure by 36% on average, with reductions of up to 88%.

Copied to clipboard

Shivam Chandhok, Wan-Cyuan Fan, Vered Shwartz, Vineeth N. Balasubramanian, Leonid Sigal

VIBE: Can a VLM Read the Room? (2025.findings-emnlp)

Challenge:	Vision-language Models have been shown to be highly capable but lacking basic visual understanding skills.
Approach:	They propose to examine the limitations of vision-language models on visual tasks by constructing a series of tests that probe which components of design may be lacking.
Outcome:	The proposed tests compare VLMs to other models on visual encoders, intermediate vision-language projection and LLM-decoder outputs.

Copied to clipboard

Tania Chakraborty, Eylon Caplan, Dan Goldwasser

ProcVQA: Benchmarking the Effects of Structural Properties in Mined Process Visualizations on Vision–Language Model Performance (2025.findings-emnlp)

Challenge:	Vision Language Models (LLMs) cannot account for the role that non-verbal cues play in understanding social situations.
Approach:	They propose a task to test the capabilities of Vision Language Models (VLMs) to account for the visual social-pragmatic inference gap.
Outcome:	The proposed task tests the capabilities of a VLM for a social reasoning task.

Copied to clipboard

Kazi Tasnim Zinat, Saad Mohammad Abrar, Shoumik Saha, Sharmila Duppala, Saimadhav Naga Sakhamuri, Zhicheng Liu

SteerVLM: Robust Model Control through Lightweight Activation Steering for Vision Language Models (2025.findings-emnlp)

Challenge:	Vision-Language Models have shown impressive capabilities and notable failures in data visualization understanding tasks.
Approach:	They propose a benchmark to analyze how specific properties within a visualization type affect VLM performance.
Outcome:	The proposed benchmark examines how specific properties affect VLM performance . it shows that models exhibit steep drops on multi-hop reasoning and extraction errors increase with edge density .

Copied to clipboard

Anushka Sivakumar, Andrew Zhang, Zaber Ibn Abdul Hakim, Chris Thomas

LPOI: Listwise Preference Optimization for Vision Language Models (2025.acl-long)

Challenge:	SteerVLM is a lightweight steering module designed to guide Vision-Language Models (VLMs) towards outputs that better adhere to desired instructions.
Approach:	They propose a lightweight steering module that learns from latent embeddings of paired prompts encoding target and converse behaviors to dynamically adjust activations connecting the language modality with image context.
Outcome:	The proposed steering module outperforms existing intervention techniques on steering and hallucination mitigation benchmarks for VLMs.

Copied to clipboard

Fatemeh Pesaran Zadeh, Yoojin Oh, Gunhee Kim

Sightation Counts: Leveraging Sighted User Feedback in Building a BLV-aligned Dataset of Diagram Descriptions (2025.acl-long)

Challenge:	Existing methods for aligning large VLMs with human preferences often overfit to textual information or exacerbate hallucinations.
Approach:	They propose an object-aware listwise preference optimization for reducing hallucinations in VLMs . they mask a critical object in an image and interpolate the masked region to form more complete images .
Outcome:	The proposed method outperforms existing methods in reducing hallucinations and enhancing performance on MMHalBench, AMBER, and Object HalBench.

Copied to clipboard

Wan Ju Kang, Eunki Kim, Na Min An, Sangryul Kim, Haemin Choi, Ki Hoon Kwak, James Thorne

M2-TabFact: Multi-Document Multi-Modal Fact Verification with Visual and Textual Representations of Tabular Data (2025.findings-acl)

Challenge:	Existing studies show that direct generation of diagram descriptions is costly and biased against blind and low-vision (BLV) users.
Approach:	They ask sighted individuals to assess diagram descriptions generated by vision-language models . they use latent supervision to guide the models with latent inference .
Outcome:	The results show that visual descriptions generated by vision-language models are effective and useful to educators who are themselves BLV and teach visually impaired learners.

Copied to clipboard

Mingyang Zhou, Lingyu Zhang, Sophia Horng, Maximillian Chen, Kung-Hsiang Huang, Shih-Fu Chang

On the Fine-Grained Planning Abilities of VLM Web Agents (2025.findings-emnlp)

Challenge:	Existing fact-checking systems that can reason over structured data are inefficient compared to humans.
Approach:	They propose a multi-modal table-based fact verification task that requires reasoning over visual and textual representations of structured data.
Outcome:	The proposed model can reason over visual and textual representations of structured data.

Copied to clipboard

Surgan Jandial, Yinong Oliver Wang, Andrea Bajcsy, Fernando De la Torre

Challenge:	Vision-Language Models (VLMs) have shown promise as web agents, yet their planning has been overlooked.
Approach:	They propose to examine VLMs’ ability to understand temporal relationships within web contexts and assess plans of actions across diverse scenarios.
Outcome:	The proposed models exhibit limited performance in the above skills and are not reliable to function as web agents.

METAL: A Multi-Agent Framework for Chart Generation with Test-Time Scaling (2025.acl-long)

Copied to clipboard

Bingxuan Li, Yiwei Wang, Jiuxiang Gu, Kai-Wei Chang, Nanyun Peng

Symmetrical Visual Contrastive Optimization: Aligning Vision-Language Models with Minimal Contrastive Images (2025.acl-long)

Challenge:	Chart generation requires strong visual design skills and precise coding capabilities that embed the desired visual properties into code.
Approach:	They propose a vision-language model-based multi-agent framework for effective automatic chart generation.
Outcome:	The proposed framework achieves a 5.2% improvement in the F1 score over the current best chart generation task.

Copied to clipboard

Shengguang Wu, Fan-Yun Sun, Kaiyue Wen, Nick Haber

From Charts to Fair Narratives: Uncovering and Mitigating Geo-Economic Biases in Chart-to-Text (2025.emnlp-main)

Challenge:	Recent studies have shown that Large Vision-Language Models (VLMs) tend to neglect image content and over-rely on language-model priors, resulting in errors in visually grounded tasks and hallucinations.
Approach:	They propose a novel finetuning objective that steers the model toward capturing important visual details and aligning them with corresponding text tokens.
Outcome:	The proposed method achieves up to 22% reduction in hallucinations and significant gains in vision-centric and general tasks while maintaining or improving the model's general abilities.

Copied to clipboard

Ridwan Mahbub, Mohammed Saidul Islam, Mir Tafseer Nayeem, Md Tahmid Rahman Laskar, Mizanur Rahman, Shafiq Joty, Enamul Hoque

Scalable Vision Language Model Training via High Quality Data Curation (2025.acl-long)

Challenge:	Existing VLMs produce more positive descriptions for high-income countries compared to middle- or low-income nations, even when country attribution is the only variable changed.
Approach:	They propose to automate the process by generating textual summaries of charts using vision-language models to understand how a country’s economic status influences the sentiment of generated summary.
Outcome:	The proposed model amplifys geo-economic biases in 6,000 chart-country pairs from six widely used vision-language models to understand how a country’s economic status influences the sentiment of generated summaries.

Copied to clipboard

Hongyuan Dong, Zijian Kang, Weijie Yin, LiangXiao LiangXiao, ChaoFeng ChaoFeng, Ran Jiao

Design Choices for Extending the Context Length of Visual Language Models (2025.acl-long)

Challenge:	SAIL-VL models achieve the highest average score in 18 widely used VLM benchmarks in our evaluation, with the 2B model takes the top position over VLMs of comparable sizes on OpenCompass 2024.
Approach:	They introduce an open-source vision language model (VLM) series that can be trained using high-quality data.
Outcome:	The proposed model achieves the highest average score in 18 widely used VLM benchmarks, with the 2B model taking the top position over VLMs of comparable sizes on OpenCompass 2024.

Copied to clipboard

Mukai Li, Lei Li, Shansan Gong, Qi Liu

CoV: Chain-of-View Prompting for Spatial Reasoning (2026.findings-acl)

Challenge:	Existing open-source Visual Language Models lack systematic exploration into extending their context length, and commercial models often provide limited details.
Approach:	They propose to extend Visual Language Models (VLMs) to 128K lengths and open-source the code, data, and models.
Outcome:	The proposed model is based on the Qwen-VL series model and is competitive with commercial model GPT-4V.

Copied to clipboard

Haoyu Zhao, Akide Liu, Zeyu Zhang, Weijie Wang, Feng Chen, Ruihan Zhu, Gholamreza Haffari, Bohan Zhuang

Low-Resource NLP

Challenge:	Embodied question answering requires collecting context that is distributed across multiple viewpoints . most recent vision–language models (VLMs) are constrained to a fixed and finite set of input views .
Approach:	They propose a training-free, test-time reasoning framework that transforms a VLM into an active viewpoint reasoner through a coarse-to-fine exploration process.
Outcome:	The proposed framework improves LLM-Match performance by 11.98% on four mainstream VLMs.

Specializing Large Models for Oracle Bone Script Interpretation via Component-Grounded Multimodal Knowledge Augmentation (2026.acl-long)

Copied to clipboard

Jianing Zhang, Runan Li, Honglin Pang, Ding Xia, Zhou Zhu, Qian Zhang, Chuntao Li, Xi Yang

VISaGE: Understanding Visual Generics and Exceptions (2025.emnlp-main)

Challenge:	Existing methods for deciphering ancient Chinese Oracle Bone Script (OBS) treat deciphering as a closed-set image recognition problem, which fails to bridge the "interpretation gap" .
Approach:	They propose a vision-language model framework that integrates a VLM and an LLM to automate a reasoning chain of component identification and knowledge retrieval.
Outcome:	The proposed framework yields more detailed and precise decipherments compared to baseline methods.

Copied to clipboard

Stella Frank, Emily Allaway

What Do Vision–Language Models Encode for Personalized Image Aesthetics Assessment? (2026.findings-acl)

Challenge:	atypical evaluation instances disrupt incontext instance understanding and in-weight conceptual knowledge.
Approach:	They propose to use a dataset to analyze atypical visual and textual images to test their models.
Outcome:	The proposed model is based on a dataset consisting of typical and exceptional images.

Copied to clipboard

Koki Ryu, Hitomi Yanaka

Glyph: Scaling Context Windows via Visual-Text Compression (2026.acl-long)

Challenge:	Personalized image aesthetics assessment (PIAA) is an important research problem with practical applications.
Approach:	They propose a vision-language model that encodes multi-level aesthetic attributes . they analyze visual representations of VLMs to examine their internal representations .
Outcome:	The proposed framework can be used to personalize images without fine-tuning . it can be implemented in a variety of image domains and architectures.

Copied to clipboard

Jiale Cheng, Yusen Liu, Xinyu Zhang, Yulin Fei, Wenyi Hong, Ruiliang Lyu, Weihan Wang, Zhe Su, Xiaotao Gu, Xiao Liu, Yushi Bai, Jie Tang, Hongning Wang, Minlie Huang

Fico: Evaluating Vision-Language Models under Visual Fidelity and Compression at Scale (2026.findings-acl)

Challenge:	Large language models (LLMs) traditionally represent text as sequences of discrete tokens . a long-context scaling problem requires processing more tokens more efficiently .
Approach:	They propose a framework that renders long texts into compact visual pages and processes them with a vision-language model.
Outcome:	The proposed framework renders long texts into compact visual pages and processes them with a vision-language model.

Copied to clipboard

Jianhong Tu, Nicholas Crispino, Kyle Montgomery, Chenguang Wang, Dawn Song

Can VLMs Predict Future States? Bootstrapping World Models from Inverse Dynamics (2026.findings-acl)

Challenge:	Visual text compression is emerging paradigm for rendering text as images for processing by vision-language models.
Approach:	They propose a benchmark to assess VLM robustness under dense visual inputs.
Outcome:	Evaluating 13 general-purpose VLMs and 3 OCR-specialized models reveals performance drops sharply under increased density or reduced resolution; cross-task transfer between OCR, NIAH, and VQA is limited; and VQ is comparatively robust because low-level details are lost before high-level semantics.

Copied to clipboard

Yifu Qiu, Yftah Ziser, Anna Korhonen, Shay B Cohen, Edoardo Ponti

E-ViC: Reasoning Beyond Text via Embodied Visual Chain for Spatial Intelligence (2026.acl-long)

Challenge:	unified vision–language models (VLMs) struggle to generate physically plausible transitions between frames from instructions.
Approach:	They find that VLMs struggle to generate physically plausible transitions between frames from instructions.
Outcome:	The proposed model outperforms state-of-the-art image editing models on Aurora-Bench . it achieves the best average human evaluation across all subsets of Aurora-bench compared with other models .

Copied to clipboard

Junbo Qi, Yi Zhang, Hanchu Ni, Che Liu, Zhimin Yao, Ruilin Yang, Xiancong Ren, Liangjian Wen, Wei Ge, Yuya Ieiri, Osamu Yoshie, Yong Dai, Xiaozhu Ju

MPTc-Bench: Measuring Cross-market Generative Ability of Vision-Language Models via Movie Poster Transcreation (2026.findings-acl)

Challenge:	Existing Vision-Language Models (VLMs) lack spatial reasoning, despite text-based CoTs . e-ViC reframes spatial intelligence as a verifiable, tool-using capability, argues a new study.
Approach:	They propose a framework that moves reasoning beyond text into the visual domain . they ground reasoning in pixel-level interactions to enable human-like "look-and-confirm" strategies .
Outcome:	The proposed framework outperforms existing Vision-Language Models with an average gain of 10.1%.

Copied to clipboard

Youyuan Lin, Yuan Li, Yahan Yu, Fei Cheng, Shin’ya Nishida, Chenhui Chu

System-Mediated Attention Imbalances Make Vision-Language Models Say Yes (2026.findings-acl)

Challenge:	Recent work adapts textual transcreation to image editing and formulates image transcreations to better match a target audience while preserving meaning.
Approach:	They propose a two-stage planner-editor pipeline in which an VLM planner specifies executable edits and an image editor renders them.
Outcome:	The proposed model can transcreate a visual asset for a different market while preserving its identity while matching market-specific design preferences and multilingual typography.

Copied to clipboard

Tsan Tsai Chan, Varsha Suresh, Anisha Saha, Michael Hahn, Vera Demberg

Nash-Pruned CredMAS: Dynamic Panel Pruning for VLM-MAS using Nash-based Selection and Doubly-Robust Credits (2026.findings-acl)

Challenge:	Existing mitigation strategies tend towards an image-centric interpretation of these imbalances, prioritising increased image attention while giving less consideration to the roles of the other modalities.
Approach:	They propose a more holistic, system-mediated account which attributes imbalances to functionally redundant system weights that reduce attention to image and textual inputs.
Outcome:	The proposed framework offers a useful empirical perspective on the yes-bias, a common form of hallucination in which VLMs indiscriminately respond ‘yes’.

Copied to clipboard

Yijia Fan, Mingyu Liu, Jing Yang, Jian Wang, Keze Wang, Jusheng Zhang

VLMGuard-R1: Proactive Safety Alignment for VLMs via Reasoning-Driven Prompt Optimization (2026.findings-acl)

Challenge:	Multi-Agent Systems (MAS) are expensive due to static panel designs, where all N agents communicate at every T round.
Approach:	They propose an economic framework that transforms agent selection into a dynamic resource allocation game.
Outcome:	The proposed system reduces token consumption by over 25% on challenging benchmarks while reducing token consumption.

Copied to clipboard

Menglan Chen, Xianghe Pang, Jingjing Dong, WenHao Wang, Yaxin Du, Siheng Chen

Through the Magnifying Glass: Adaptive Perception Magnification for Hallucination-Free VLM Decoding (2026.acl-long)

Challenge:	integrating vision and language models with safety standards is essential to mitigate multimodal complexity . integrating visual inputs with vision and text unveils subtle threats beyond the reach of conventional safeguards .
Approach:	They propose a framework that combines vision and language to provide a multimodal reasoning-driven prompt rewriting.
Outcome:	The proposed framework outperforms baseline models on five benchmarks with six VLMs.

Copied to clipboard

Shunqi Mao, Chaoyi Zhang, Weidong Cai

Does RLVR Extend Reasoning Boundaries? Investigating Capability Expansion in Vision-Language Models (2026.acl-long)

Challenge:	Existing vision-language models suffer from visual hallucination, where the generated responses contain inaccuracies that are not grounded in the visual input.
Approach:	They propose a visual decoding method that iteratively isolates relevant visual tokens based on attention and magnifies the corresponding regions.
Outcome:	The proposed method reduces language biases and amplifies weights of visual embedding during decoding, while still preserving strong reasoning capabilities.

Copied to clipboard

Minghe Shen, Zhuo Zhi, Chonghan Liu, Shuo Xing, Zhengzhong Tu, Che Liu