Targeted Multilingual Adaptation for Low-resource Language Families (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Massively multilingual models are known to have limited utility in any one language, and to perform poorly on low-resource languages. |
| Approach: | They propose to adapt a pre-trained multilingual model to a language family and evaluate its performance on two downstream tasks and 11 evaluation languages. |
| Outcome: | The proposed model outperforms mono- and multilingual models on two downstream tasks and 11 evaluation languages. |
Similar Papers
Multilingual Encoder Knows more than You Realize: Shared Weights Pretraining for Extremely Low-Resource Languages (2025.acl-long)
Copied to clipboard
| Challenge: | XLM-R and mBART have advanced multilingualism in NLP, but low-resource languages such as Tibetan, Uyghur, Kazakh, and Mongolian are underserved. |
| Approach: | They propose a framework for adapting multilingual encoders to text generation in extremely low-resource languages by reusing the weights between the encoder and the decoder. |
| Outcome: | The proposed framework performs better on various downstream tasks even when compared with much larger models. |
When Is Multilinguality a Curse? Language Modeling for 250 High- and Low-Resource Languages (2024.emnlp-main)
Copied to clipboard
| Challenge: | Multilingual language models are widely used to extend NLP systems to low-resource languages. |
| Approach: | They pre-train over 10,000 monolingual and multilingual language models for over 250 languages including multiple language families that are under-studied in NLP. |
| Outcome: | The results show that adding multilingual data improves low-resource language modeling performance, similar to increasing low-source dataset sizes by up to 33%. |
Phylogeny-Inspired Adaptation of Multilingual Models to New Languages (2022.aacl-main)
Copied to clipboard
| Challenge: | Large pretrained multilingual models have delivered promising results due to cross-lingual learning capabilities on a variety of language tasks. |
| Approach: | They propose to use language phylogenetic information to improve cross-lingual transfer by leveraging closely related languages in a structured, linguistically-informed manner. |
| Outcome: | The proposed model significantly improves on the baseline model on languages unseen during training. |
Small Models, Big Impact: Efficient Corpus and Graph-Based Adaptation of Small Multilingual Language Models for Low-Resource Languages (2025.acl-srw)
Copied to clipboard
| Challenge: | Low-resource languages (LRLs) face significant challenges in natural language processing due to limited data. |
| Approach: | They evaluate adapter-based methods for adapting mLMs to low-resource languages . they use unstructured text and structured knowledge from ConceptNet to evaluate adapters . |
| Outcome: | The proposed methods outperform large language models and LLaMA-3 and deepSeek-R1 models on low training data. |
The Less the Merrier? Investigating Language Representation in Multilingual Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Multilingual models can be used to integrate multiple languages into one model and use cross-language transfer learning to improve performance for different NLP tasks. |
| Approach: | They propose to include languages in popular multilingual models and to use cross-language transfer learning to improve performance for different NLP tasks. |
| Outcome: | The proposed models perform better on downstream tasks for seen and unseen languages than community-centered models for low-resource languages. |
Instructing Large Language Models for Low-Resource Languages: A Systematic Study for Basque (2025.emnlp-main)
Copied to clipboard
Oscar Sainz, Naiara Perez, Julen Etxaniz, Joseba Fernandez de Landa, Itziar Aldabe, Iker García-Ferrero, Aimar Zabala, Ekhi Azurmendi, German Rigau, Eneko Agirre, Mikel Artetxe, Aitor Soroa
| Challenge: | Instructing language models with user intent requires large instruction datasets limited to a limited set of languages. |
| Approach: | They propose to use existing LLMs and synthetically generated instructions to train models with user intent. |
| Outcome: | The proposed model outperforms base non-instructed models on Basque without Basque instructions. |
Systematic Investigation of Strategies Tailored for Low-Resource Settings for Low-Resource Dependency Parsing (2023.eacl-main)
Copied to clipboard
| Challenge: | Several strategies have been proposed to enhance performance in low-resource scenarios. |
| Approach: | They propose to use 5 low-resource strategies for dependency parsing for multiple languages . they use ensembled approach on 7 UD low-rsource languages based on their results . |
| Outcome: | The proposed approach improves on a low-resource language Sanskrit. |
Parsing with Multilingual BERT, a Small Corpus, and a Small Treebank (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Pretrained multilingual contextual representations have shown great success, but their benefits do not apply equally to all language varieties. |
| Approach: | They propose to use additional language-specific pretraining and vocabulary augmentation to adapt multilingual models to low-resource settings. |
| Outcome: | The proposed methods significantly improve performance over baselines, especially in the lowest-resource cases, and demonstrate the importance of the relationship between such models’ pretraining data and target language varieties. |
Mini But Mighty: Efficient Multilingual Pretraining with Linguistically-Informed Data Selection (2023.findings-eacl)
Copied to clipboard
| Challenge: | AfriBERTa shows that training transformer models from scratch on 1GB of data from many unrelated African languages outperforms massively multilingual models on downstream NLP tasks. |
| Approach: | They propose that training on smaller amounts of data but from related languages could match the performance of models trained on large, unrelated data. |
| Outcome: | The proposed model outperforms models trained on large, unrelated datasets on downstream NLP tasks. |
UNKs Everywhere: Adapting Multilingual Language Models to New Scripts (2021.emnlp-main)
Copied to clipboard
| Challenge: | Massively multilingual language models offer state-of-the-art cross-lingual transfer performance on a range of NLP tasks, but there is a profound performance gap between resource-rich and resource-poor target languages. |
| Approach: | They propose a series of data-efficient methods that enable quick and effective adaptation of pretrained multilingual models to low-resource languages and unseen scripts. |
| Outcome: | The proposed methods improve learning of the new dedicated embedding matrix in the target language and for low-resource languages written in unseen scripts. |