Towards Cross-Lingual Explanation of Artwork in Large-scale Vision Language Models (2025.findings-naacl)
Copied to clipboard
| Challenge: | LVLMs are increasingly capable of responding in multiple languages . however, there is a lack of evaluation tools for LVLs that handle multiple languages. |
| Approach: | They used an extended dataset in multiple languages to evaluate LVLMs' ability to generate explanations in multiple language combinations. |
| Outcome: | The proposed dataset in multiple languages evaluates LVLMs' ability to generate explanations in other languages. |
Similar Papers
Centurio: On Drivers of Multilingual Ability of Large Vision-Language Model (2025.acl-long)
Copied to clipboard
Gregor Geigle, Florian Schneider, Carolin Holtermann, Chris Biemann, Radu Timofte, Anne Lauscher, Goran Glavaš
| Challenge: | Existing models for large vision-language tasks are trained on English data, which makes them struggle to understand non-English input and fail to generate output in the desired target language. |
| Approach: | They conduct multi-stage experiments on 13 vision-language tasks and 43 languages . they find that one can include as many as 100 training languages simultaneously with as little as 25-50% of non-English data . |
| Outcome: | The proposed model outperforms existing models in 14 tasks and 56 languages. |
Towards Artwork Explanation in Large-scale Vision Language Models (2024.acl-short)
Copied to clipboard
| Challenge: | Large-scale Vision-Language Models (LVLMs) output text from images and instructions, demonstrating advanced capabilities in text generation and comprehension. |
| Approach: | They propose to use artwork explanation generation task to quantitatively assess the understanding and utilization of artworks knowledge. |
| Outcome: | The proposed task evaluates the understanding and utilization of knowledge about artworks from images and titles and generates explanations using only images. |
Finer: Investigating and Enhancing Fine-Grained Visual Concept Recognition in Large Vision Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in instruction-tuned Large Vision-Language Models (LVLMs) have imbued the models with the ability to generate high-level, image-grounded explanations with ease. |
| Approach: | They propose to use a multiple granularity attribute-centric benchmark and training mixture to evaluate LVLMs’ fine-grained visual comprehension ability. |
| Outcome: | The proposed model improves on LLaVa-1.5, InstructBLIP and GPT-4V and demonstrates that they struggle to generate descriptive visual attributes based on a concept that appears within an input image despite their prominent zero-shot image captioning ability. |
CLAIM: Mitigating Multilingual Object Hallucination in Large Vision-Language Models with Cross-Lingual Attention Intervention (2025.acl-long)
Copied to clipboard
Zekai Ye, Qiming Li, Xiaocheng Feng, Libo Qin, Yichong Huang, Baohang Li, Kui Jiang, Yang Xiang, Zhirui Zhang, Yunfei Lu, Duyu Tang, Dandan Tu, Bing Qin
| Challenge: | Large Vision-Language Models (LVLMs) have impressive multimodal abilities but remain prone to multilingual object hallucination. |
| Approach: | They propose a cross-lingual attention intervention method to mitigate multilingual object hallucination in LVLMs by aligning attention patterns. |
| Outcome: | The proposed method improves 13.56% (up to 30%) on the POPE and 21.75% on the hallucination subsets across languages. |
LLMs Beyond English: Scaling the Multilingual Capability of LLMs with Cross-Lingual Feedback (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent multilingual models support limited number of human languages due to lack of training data for low resource languages. |
| Approach: | They propose a multilingual multilingual LLM that scales to 100 languages . they use a human feedback dataset and a data set to perform multilingual instruction tuning . |
| Outcome: | The proposed model outperforms its peers on five multilingual benchmarks. |
Are Large Vision Language Models up to the Challenge of Chart Comprehension and Reasoning (2024.findings-emnlp)
Copied to clipboard
Mohammed Saidul Islam, Raian Rahman, Ahmed Masry, Md Tahmid Rahman Laskar, Mir Tafseer Nayeem, Enamul Hoque
| Challenge: | Recent studies have demonstrated that large vision language models (LVLMs) are not multi-modal and lack multi-tasking capabilities. |
| Approach: | They evaluate the performance of large vision language models (LVLMs) for chart understanding and reasoning tasks and compare them to open-source models. |
| Outcome: | The proposed models demonstrate impressive abilities in generating fluent texts covering high-level data insights, but they also encounter common problems like hallucinations, factual errors, and data bias. |
Language-Specific Layer Matters: Efficient Multilingual Enhancement for Large Vision-Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large vision-language models exhibit an imbalance in multilingual capabilities . |
| Approach: | They propose a training recipe that achieves efficient multilingual enhancement for LVLMs by Precise Language Specific layers fine-tuning. |
| Outcome: | The proposed training recipe achieves efficient multilingual enhancement for LVLMs by fine-tuning language specific layers. |
More Images, More Problems? A Controlled Analysis of VLM Failure Modes. (2026.findings-acl)
Copied to clipboard
Anurag Das, Adrian Bulat, Alberto Baldrati, Ioannis Maniadis Metaxas, Bernt Schiele, Georgios Tzimiropoulos, Brais Martinez
| Challenge: | Existing evaluations of large vision language models lack a comprehensive analysis of their weaknesses and causes. |
| Approach: | They propose a new benchmark to evaluate multi-image capabilities of Large Vision Language Models. |
| Outcome: | The proposed model outperforms existing benchmarks on multi-image models. |
Benchmarking and Improving Large Vision-Language Models for Fundamental Visual Graph Understanding and Reasoning (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for enhancing understanding and reasoning abilities in graphbased tasks focus on specific graph types or tasks, posing challenges in designing versatile systems suitable for various tasks and graphs across diverse domains. |
| Approach: | They propose a structure-aware fine-tuning framework to enhance LVLMs with structure learning abilities through three self-supervised learning tasks. |
| Outcome: | Extensive evaluations on 14 LVLMs reveal that LVLs are weak in basic graph understanding and reasoning tasks, particularly those concerning relational or structurally complex information. |
Unlocking Multilingual Reasoning Capability of LLMs and LVLMs through Representation Engineering (2026.acl-long)
Copied to clipboard
Qiming Li, Xiaocheng Feng, Yixuan Ma, Ruihan Chen, Zihe Tong, Zekai Ye, Xiachong Feng, Libo Qin, Haoyu Ren, Kun Chen, Yunfei Lu, Dandan Tu, Bing Qin
| Challenge: | Existing approaches to enhance multilingual reasoning capabilities rely on costly multilingual training or employ prompting with external translation tools. |
| Approach: | They propose a training-free inference-time method to enhance multilingual reasoning capabilities via Representation Engineering without additional training data or tools. |
| Outcome: | The proposed method outperforms existing methods on four reasoning benchmarks in English and Thai and Swahili. |