Annotation-Efficient Vision-Language Model Adaptation to the Polish Language Using the LLaVA Framework (2026.eacl-srw)
Copied to clipboard
Grzegorz Statkiewicz, Alicja Dobrzeniecka, Karolina Seweryn, Aleksandra Krasnodębska, Karolina Piosek, Katarzyna Bogusz, Sebastian Cygert, Wojciech Kusa
| Challenge: | Currently, most vision-language models are trained on English-centric data, limiting their usability for non-English-speaking users. |
| Approach: | They reproduce and adapt LLaVA-Next methodology to create Polish VLMs . they use a fully automated pipeline for translating and filtering existing multimodal datasets based on Polish data for OCR and culturally specific tasks. |
| Outcome: | The proposed model improves on a Polish-adapted model and shows higher quality captions in generative evaluations. |
Similar Papers
Why do LLaVA Vision-Language Models Reply to Images in English? (2024.findings-emnlp)
Copied to clipboard
Musashi Hinck, Carolin Holtermann, Matthew Olson, Florian Schneider, Sungduk Yu, Anahita Bhiwandiwalla, Anne Lauscher, Shao-Yen Tseng, Vasudev Lal
| Challenge: | Including an image in a multimodal query significantly increases the likelihood of the model returning an English response regardless of the language of the query. |
| Approach: | They propose a two-pronged approach that combines extensive ablation of the design space with a mechanistic analysis of the models’ internal representations of image and text inputs. |
| Outcome: | The proposed approach reduces the multilingual error by switching the language backbone for a bilingual language model. |
VFA: Empowering Multilingual MLLMs via Vision-Free Adaptation (2026.acl-long)
Copied to clipboard
Yixia Li, Yaqing Shi, Zhiwen Ruan, Dongdong Zhang, Lingjie Jiang, Shaohan Huang, Yun Chen, Guanhua Chen, Furu Wei
| Challenge: | Multimodal large language models have advanced rapidly, yet most remain English-centric . scaling multilingual multimodal instruction tuning is limited by the scarcity and high cost of non-English image–text supervision. |
| Approach: | They propose a framework that decouples multilingual language enhancement from visual alignment by composing complementary task vectors over a shared LLM backbone. |
| Outcome: | The proposed framework achieves competitive performance with a fully multimodally trained model using less than 2% of the text data. |
X-LLaVA: Optimizing Bilingual Large Vision-Language Alignment (2024.findings-naacl)
Copied to clipboard
DongJae Shin, HyeonSeok Lim, Inho Won, ChangSu Choi, Minjun Kim, SeungWoo Song, HanGyeol Yoo, SangMin Kim, KyungTae Lim
| Challenge: | constructing multilingual data for large multimodal models presents its own set of challenges due to language diversity and complexity. |
| Approach: | They propose to use GPT4-V to construct multimodal training datasets using a text-only version of GPT4. |
| Outcome: | The proposed method performs well in Korean and English, surpassing existing methods. |
Recovered in Translation: Efficient Pipeline for Automated Translation of Benchmarks and Datasets (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks suffer from semantic drift and context loss, which can lead to misleading performance metrics. |
| Approach: | They propose a fully automated framework to enable translation of large language models . they propose to use universal self-improvement and multi-round ranking methods to improve translation quality . |
| Outcome: | The proposed framework surpasses existing benchmarks in eight languages and improves translation quality across multilingual domains. |
Expedited Training of Visual Conditioned Language Generation via Redundancy Reduction (2024.acl-long)
Copied to clipboard
| Challenge: | EVLGen is a framework for visual-language pre-training with high computational demands. |
| Approach: | They propose a streamlined framework for the pre-training of visually conditioned language generation models with high computational demands. |
| Outcome: | The proposed framework accelerates training of vision-language models by a factor of 5 without compromising performance. |
Language Adaptation of Large Language Models: An Empirical Study on LLaMA2 (2025.coling-main)
Copied to clipboard
| Challenge: | Popularity of Large Language Models (LLMs) has seen a skyrocketing increase in recent years. |
| Approach: | They present a systematic review of the language adaptation process for Large Language Models including vocabulary expansion, continued pre-training, and instruction fine-tuning. |
| Outcome: | The proposed model is based on empirical studies conducted on LLaMA2 and discussions on various settings affecting the model's capabilities. |
VLIS: Unimodal Language Models Guide Multimodal Language Generation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing vision-language models face challenges in tasks that require complex linguistic understanding. |
| Approach: | They propose a framework that combines visual conditioning and linguistic understanding of unimodal text-only language models without further training to improve vision-language models. |
| Outcome: | The proposed framework improves vision-language models on diverse tasks including commonsense understanding and complex text generation. |
Llama2Vec: Unsupervised Adaptation of Large Language Models for Dense Retrieval (2024.acl-long)
Copied to clipboard
| Challenge: | Dense retrieval requires discriminative embeddings to represent the semantic relationship between query and document. |
| Approach: | They propose an unsupervised approach that performs unsupervised adaptation of large language models for dense retrieval. |
| Outcome: | The proposed model improves on a variety of dense retrieval benchmarks and is available on github. |
LLaVA Steering: Visual Instruction Tuning with 500x Fewer Parameters through Modality Linear Representation-Steering (2025.acl-long)
Copied to clipboard
| Challenge: | Multimodal Large Language Models (MLLMs) enhance visual tasks by integrating visual representations into large language models. |
| Approach: | They propose a method to re-balance modalities by steering visual representations . they propose LLaVA Steering, a platform that enables rapid customization of MLLMs a component-based architecture . |
| Outcome: | The proposed model re-balances the modalities of visual representations in large language models . the model requires 500 times fewer trainable parameters than LoRA while maintaining comparable performance . |
Data and Model Centric Approaches for Expansion of Large Language Models to New languages (2025.emnlp-tutorials)
Copied to clipboard
| Challenge: | Existing LLMs mainly support English alongside a handful of high resource languages . this leaves a major gap for most low-resource languages despite increasing pace of research . |
| Approach: | This tutorial examines approaches to expand the language coverage of LLMs . they look at tokenizer training, pre-training, instruction tuning, alignment, evaluation, etc. |
| Outcome: | This tutorial examines approaches to expand the language coverage of LLMs . it provides an efficient and viable path to bring LLM technologies to low-resource languages . |