Pipeline Analysis for Developing Instruct LLMs in Low-Resource Languages: A Case Study on Basque (2025.naacl-long)
Copied to clipboard
| Challenge: | Large language models are typically optimized for resource-rich languages like English . however, the proprietary nature of these models makes them impractical for many researchers and developers. |
| Approach: | They propose to develop large language models that can follow instructions in Basque . they focus on three key stages: pre-training, instruction tuning, and alignment with human preferences . |
| Outcome: | The proposed models improve natural language understanding (NLU) of the foundational model by 12 points . the results show that the models can follow instructions in Basque with human preferences . |
Similar Papers
Instructing Large Language Models for Low-Resource Languages: A Systematic Study for Basque (2025.emnlp-main)
Copied to clipboard
Oscar Sainz, Naiara Perez, Julen Etxaniz, Joseba Fernandez de Landa, Itziar Aldabe, Iker García-Ferrero, Aimar Zabala, Ekhi Azurmendi, German Rigau, Eneko Agirre, Mikel Artetxe, Aitor Soroa
| Challenge: | Instructing language models with user intent requires large instruction datasets limited to a limited set of languages. |
| Approach: | They propose to use existing LLMs and synthetically generated instructions to train models with user intent. |
| Outcome: | The proposed model outperforms base non-instructed models on Basque without Basque instructions. |
LLMs for Extremely Low-Resource Finno-Ugric Languages (2025.findings-naacl)
Copied to clipboard
| Challenge: | Low-resource languages such as those in the Finno-Ugric family are underrepresented in large language models. |
| Approach: | They propose to develop large language models for extremely low-resource languages . they focus on Vro, Livonian, and Komi, which are underrepresented . |
| Outcome: | The proposed models cover almost the entire cycle of creation, from data collection to instruction tuning and evaluation. |
Machine Translation for Low-Resource Languages through Monolingual Data and LLM: A Case Study of English-to-Basque (2026.eacl-srw)
Copied to clipboard
| Challenge: | Existing LLMs do not translate well from English to Basque, but they yield an acceptable performance in the reverse direction. |
| Approach: | They propose to use a Basque monolingual corpora to train an LLM-based MT system . they use 'sovereignty fine tuning' to generate parallel corporata, and then use preference optimization . |
| Outcome: | The proposed system improves translation quality in English-to-Basque direction while requiring limited data for low-resource languages. |
Optimizing Language Augmentation for Multilingual Large Language Models: A Case Study on Korean (2024.lrec-main)
Copied to clipboard
ChangSu Choi, Yongbin Jeong, Seoyoon Park, Inho Won, HyeonSeok Lim, SangMin Kim, Yejee Kang, Chanhyuk Yoon, Jaewan Park, Yiseul Lee, HyeJin Lee, Younggyun Hahm, Hansaem Kim, KyungTae Lim
| Challenge: | Large language models (LLMs) use pretraining to predict the subsequent word, but less-resourced languages are being overlooked. |
| Approach: | They propose to expand the MLLM vocabularies to enhance expressiveness and use bilingual data for pretraining to align the high- and less-resourced languages. |
| Outcome: | The proposed model outperforms existing models in qualitative analyses compared to Korean monolingual models. |
Extending LLMs to New Languages: A Case Study of Llama and Persian Adaptation (2025.coling-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) are mainly trained on English data and struggle with low-resource languages. |
| Approach: | They propose to add a new language to Llama to improve classification accuracy for Persian tasks by aligning representations through bilingual pretraining and instruction datasets. |
| Outcome: | The proposed model performs on generation and classification tasks with no adverse impact and sometimes even improvements on English tasks. |
INTERS: Unlocking the Power of Large Language Models in Search with Instruction Tuning (2024.acl-long)
Copied to clipboard
Yutao Zhu, Peitian Zhang, Chenghao Zhang, Yifei Chen, Binyu Xie, Zheng Liu, Ji-Rong Wen, Zhicheng Dou
| Challenge: | Large language models (LLMs) have demonstrated impressive capabilities in various natural language processing tasks, but their application to information retrieval tasks is still challenging due to the infrequent occurrence of many IR-specific concepts in natural language. |
| Approach: | They propose to use instruction tuning to enhance LLMs' proficiency in IR tasks by combining a dataset with manually written templates to analyze the effects of instruction design, template diversity, few-shot demonstrations, and the volume of instructions. |
| Outcome: | The proposed model can be used to perform query understanding, document understanding, and query-document relationship understanding tasks. |
LLMs for Low Resource Languages in Multilingual, Multimodal and Dialectal Settings (2024.eacl-tutorials)
Copied to clipboard
| Challenge: | Recent advances in AI can be attributed to the remarkable performance of Large Language Models (LLMs) success of LLMs depends on specific training techniques, such as instruction tuning and prompting . |
| Approach: | They explore the capabilities of Large Language Models (LLMs) in various tasks and languages . they also examine their performance, fine-tuning, instructions tuning, and close vs. open models . |
| Outcome: | The proposed model can be used for speech and multimodal tasks across modalities, languages, and dialects. |
VEEF-Multi-LLM: Effective Vocabulary Expansion and Parameter Efficient Finetuning Towards Multilingual Large Language Models (2025.coling-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have a significant disadvantage for low-resource languages . VEEF-Multi-LLM-8B excels in multilingual instruction-following tasks . |
| Approach: | They propose a low-resource multilingual large language model that expands the vocabulary for multilingual support. |
| Outcome: | The proposed model outperforms existing models in multilingual instruction-following tasks, but lags behind English-centric models in some tasks. |
Walia-LLM: Enhancing Amharic-LLaMA by Integrating Task-Specific and Generative Datasets (2024.findings-emnlp)
Copied to clipboard
Israel Azime, Atnafu Tonja, Tadesse Belay, Mitiku Yohannes Fuge, Aman Wassie, Eyasu Jada, Yonas Chanie, Walelign Sewunetie, Seid Yimam
| Challenge: | Low-resource languages are left behind due to the unavailability of resources. |
| Approach: | They propose to integrate task-specific and generative datasets to improve language model performance for Amharic by fine-tuning an Amharican instruction fine-to-tuned model. |
| Outcome: | The proposed model shows promising results in different NLP tasks and compares translated instruction datasets with the original model. |
Data and Model Centric Approaches for Expansion of Large Language Models to New languages (2025.emnlp-tutorials)
Copied to clipboard
| Challenge: | Existing LLMs mainly support English alongside a handful of high resource languages . this leaves a major gap for most low-resource languages despite increasing pace of research . |
| Approach: | This tutorial examines approaches to expand the language coverage of LLMs . they look at tokenizer training, pre-training, instruction tuning, alignment, evaluation, etc. |
| Outcome: | This tutorial examines approaches to expand the language coverage of LLMs . it provides an efficient and viable path to bring LLM technologies to low-resource languages . |