Papers with LRL
FAD-X: Fusing Adapters for Cross-lingual Transfer to Low-Resource Languages (2022.aacl-short)
Copied to clipboard
| Challenge: | Adapter-based tuning is a technique that selectively updates language-specific parameters to adapt to a new language, rather than fine-tuning all shared weights. |
| Approach: | They propose to add light-weight adapters to multilingual pretrained language models (mPLMs) and add language-specific parameters to adapt to a new language. |
| Outcome: | The proposed adapter can enhance cross-lingual transfer from pretrained adapters for well-known named entity recognition and classification benchmarks. |
Overlap-based Vocabulary Generation Improves Cross-lingual Transfer Among Related Languages (2022.acl-long)
Copied to clipboard
| Challenge: | Pre-trained multilingual models have shown great potential for zero-shot cross-lingual transfer to low web-resource languages (LRLs). |
| Approach: | They propose a vocabulary generation algorithm which enhances lexical overlap across related languages by generating a token that increases the representation of LRLs. |
| Outcome: | The proposed approach improves cross-lingual transfer accuracy without reducing HRL representation and accuracy. |
On the effective transfer of knowledge from English to Hindi Wikipedia (2025.coling-industry)
Copied to clipboard
| Challenge: | Existing studies show that in low-resource languages, Wikipedia articles on the same topic differ significantly due to cultural relevance and the varied expertise of contributors. |
| Approach: | They propose a lightweight framework to enhance knowledge equity between English and Hindi Wikipedia sections by extracting relevant information from external resources readily available. |
| Outcome: | The proposed framework enhances Hindi Wikipedia articles by 65% and 62% based on automatic and human judgment-based evaluations. |
Language, OCR, Form Independent (LOFI) pipeline for Industrial Document Information Extraction (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Existing models for low-resource language (LRL) documents are limited in semantic entity extraction, and there are limitations in SER from word level results. |
| Approach: | They propose a pipeline for Document Information Extraction (DIE) in low-resource language (LRL) business documents that solves language, Optical Character Recognition (OCR), and form dependencies through flexible model architecture, a token-level box split algorithm, and the SPADE decoder. |
| Outcome: | Experiments on Korean and Japanese documents show that the pipeline performs well in the Semantic Entity Recognition task without pre-training. |
Rapid Adaptation of Neural Machine Translation to New Languages (D18-1)
Copied to clipboard
| Challenge: | Existing approaches to adapt neural machine translation systems to low-resource languages are difficult to implement and require large amounts of training data. |
| Approach: | They propose a method to train neural machine translation systems to new low-resource languages . they propose to start with massively multilingual "seed models" and continue training on data related to the LRL . |
| Outcome: | The proposed method achieves BLEU scores of up to 15.5 with no data from the LRL and improves over other adaptation methods by 1.7 BLUE points average over 4 LRL settings. |
Exploiting Language Relatedness for Low Web-Resource Language Model Adaptation: An Indic Languages Study (2021.acl-long)
Copied to clipboard
Yash Khemchandani, Sarvesh Mehtani, Vaidehi Patil, Abhijeet Awasthi, Partha Talukdar, Sunita Sarawagi
| Challenge: | Recent research in multilingual language models (LMs) has demonstrated their ability to effectively handle multiple languages in a single model. |
| Approach: | They propose to exploit relatedness among languages in a language family to overcome corpora limitations of LRLs. |
| Outcome: | The proposed model exploits relatedness among languages in a language family to overcome corpora limitations for low web-resource languages. |
Rethinking Vocabulary Augmentation: Addressing the Challenges of Low-Resource Languages in Multilingual Models (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods to augment vocabularies ignore the disparities between model representation and frequency distributions. |
| Approach: | They propose an Entropy-Consistency Word Selection method which integrates semantic and frequency metrics for vocabulary augmentation. |
| Outcome: | The proposed method improves performance for low-resource languages compared to high-resourced ones . it integrates semantic and frequency metrics for vocabulary augmentation . |
Efficient Low-Resource Language Adaptation via Multi-Source Dynamic Logit Fusion (2026.acl-long)
Copied to clipboard
| Challenge: | Proxy Tuning offers a logit-level strategy for introducing scaling effects, but it often fails in LRL settings because the large model’s weak LRL competence might overwhelm the knowledge of specialized smaller models. |
| Approach: | They propose a logit-based framework that balances LRL competence from a continually pretrained small model, task competence from high-resource language instruction tuning, and the scaling benefits of large models. |
| Outcome: | Experiments across four model families and eight LRLs show that TriMix outperforms single-model baselines and Proxy Tuning. |
Competence-based Curriculum Learning for Multilingual Machine Translation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing multilingual machine translation models face an imbalance problem due to the different learning competencies of different languages. |
| Approach: | They propose Competence-based Curriculum Learning for Multilingual Machine Translation, named CCL-M, to help schedule the high resource languages and low resource languages. |
| Outcome: | The proposed approach achieves a steady and significant performance gain compared to the previous state-of-the-art approach on the TED talks dataset. |
Breaking Down Multilingual Machine Translation (2022.findings-acl)
Copied to clipboard
| Challenge: | Multilingual training is an essential ingredient in machine translation systems . but it has different effects in different multilingual settings, such as many-to-one, one-tomany and many- to-many learning . |
| Approach: | They compare multilingual training settings with encoders and decoders initialized by multilingual learning . they find important attention heads for each language pair and compare their correlations during inference . |
| Outcome: | The proposed models outperform the best models for high-resource languages and one-to-many models for low-resourced languages. |
Neural Factor Graph Models for Cross-lingual Morphological Tagging (P18-1)
Copied to clipboard
| Challenge: | Existing approaches to morphological tagging are limited by the assumption that tag sets overlap . a limited amount of data is available for most languages to learn these morphology taggers. |
| Approach: | They propose a method for cross-lingual morphological tagging that relaxes this assumption . they use factorial conditional random fields with neural network potentials to smooth over superficial differences in the surface forms . |
| Outcome: | The proposed model can smooth over superficial differences in the surface forms and generate unseen or rare tag sets. |
Unlocking Parameter-Efficient Fine-Tuning for Low-Resource Language Translation (2024.findings-naacl)
Copied to clipboard
| Challenge: | Parameter-efficient fine-tuning (PEFT) methods are important in low-resource language (LRL) Neural Machine Translation (NMT) but their practical effectiveness varies significantly across different languages. |
| Approach: | They evaluated the performance of 8 parameters-efficient fine-tuning methods with 15 architectures using the SacreBLEU score. |
| Outcome: | The Houlsby+Inversion adapter outperforms the baseline architectures in both in-domain and out-domain tests and the Houlson+Inverter achieves the best performance overall. |
Improving Target-side Lexical Transfer in Multilingual Neural Machine Translation (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Multilingual data is more beneficial for NMT models that translate from the LRL to a target language than those that translate into the LLLs. |
| Approach: | They propose a decoder that embeds character n-grams into NMT models that translate from an LRL to a target language. |
| Outcome: | The proposed decoder improves the performance of NMT models that translate from an LRL to a target language. |
Cross-Lingual Transfer Robustness to Lower-Resource Languages on Adversarial Datasets (2024.lrec-main)
Copied to clipboard
| Challenge: | Multilingual Language Models (MLLMs) exhibit robust cross-lingual transfer capabilities for downstream tasks such as Named Entity Recognition (NER) challenges persist in MLLM implementations that are not cross-linguistically robust. |
| Approach: | They evaluate two well-known MLLMs on 13 pairs of languages with a geographic, genetic, or borrowing relationship. |
| Outcome: | The proposed models show that they can leverage information acquired in a source language and apply it to a target language. |
Disentangling Pretrained Representation to Leverage Low-Resource Languages in Multilingual Machine Translation (2024.lrec-main)
Copied to clipboard
| Challenge: | Multilingual neural machine translation requires an enormous dataset, leaving the low-resource language (LRL) underdeveloped. |
| Approach: | They evaluated five languages using a parallel corpus of 1,000 instances each and found a zero-shot improvement of 7.4 from the baseline score of 7.1 to a score of 15.5 at best. |
| Outcome: | The proposed model improves performance in the linguistically diverse country of Indonesia by 7.4 from baseline score of 7.1 to 15.5 at best. |
Design Challenges in Low-resource Cross-lingual Entity Linking (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing techniques for grounding mentions of entities in a foreign language do not rise to the challenges introduced by text in low-resource languages (LRL) and fail to generalize to text not taken from Wikipedia, on which they are usually trained. |
| Approach: | They propose a cross-lingual XEL technique that uses search engines to locate and search for foreign language entries in Wikipedia. |
| Outcome: | The proposed system shows an increase of 25% in gold candidate recall and 13% in end-to-end linking accuracy over state-of-the-art baselines. |
Entropy-guided Vocabulary Augmentation of Multilingual Language Models for Low-resource Tasks (2023.findings-acl)
Copied to clipboard
| Challenge: | Multilingual language models (MLLMs) support low-resource languages (LRLs) but LRL words are under-represented in wordpiece/subword vocabularies, leading to low task accuracy . |
| Approach: | They propose an entropy-based vocabulary augmented language model to detect LRL words with undesirable wordpiece segmentations. |
| Outcome: | The proposed model improves performance and limits on wordpiece augmentation strategies for multiple diverse LRLs. |
MiLiC-Eval: Benchmarking Multilingual LLMs for China’s Minority Languages (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models excel in high-resource languages but struggle with low-resourced languages . minority languages such as Tibetan, Uyghur, Kazakh, and Mongolian are marginalized in NLP research due to limited digital representation and the scarcity of training data. |
| Approach: | They propose a benchmark for minority languages in China that tracks the progress of large language models on low-resource languages. |
| Outcome: | The proposed benchmark focuses on underrepresented writing systems and syntax-intensive tasks. |
Learning Latent Relations for Temporal Knowledge Graph Reasoning (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods for Temporal Knowledge Graph reasoning capture intra- and inter-time latent relations between entities that appear at different times. |
| Approach: | They propose a Latent relations Learning method for TKG reasoning that captures latent relations between entities at different times. |
| Outcome: | The proposed method exploits the intra- and inter-time latent relations of entities at different times. |
AxomiyaBERTa: A Phonologically-aware Transformer Model for Assamese (2023.findings-acl)
Copied to clipboard
| Challenge: | AxomiyaBERTa is a novel BERT model for low-resource languages . Transformers require extensive computing resources and suffer in low-compute settings . |
| Approach: | They propose a novel BERT model for Assamese, a morphologically-rich low-resource language of eastern India that is trained on a simple masked language modeling task without the NSP objective. |
| Outcome: | The proposed model performs well on token-level tasks and on “longer context” tasks with the aid of embedding disperser and phonological signals. |
Cost-Performance Optimization for Processing Low-Resource Language Tasks Using Commercial LLMs (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) exhibit impressive zero/few-shot inference and generation quality for high-resource languages (HRLs). |
| Approach: | They propose to reduce the cost of processing LRLs by code-mixing, translation, and transliteration of LRL to HRLs to ensure that predictive and generative qualities are not compromised. |
| Outcome: | The proposed model reduces the cost of processing LRLs while ensuring that predictive and generative qualities are not compromised. |
Optimizing Language Augmentation for Multilingual Large Language Models: A Case Study on Korean (2024.lrec-main)
Copied to clipboard
ChangSu Choi, Yongbin Jeong, Seoyoon Park, Inho Won, HyeonSeok Lim, SangMin Kim, Yejee Kang, Chanhyuk Yoon, Jaewan Park, Yiseul Lee, HyeJin Lee, Younggyun Hahm, Hansaem Kim, KyungTae Lim
| Challenge: | Large language models (LLMs) use pretraining to predict the subsequent word, but less-resourced languages are being overlooked. |
| Approach: | They propose to expand the MLLM vocabularies to enhance expressiveness and use bilingual data for pretraining to align the high- and less-resourced languages. |
| Outcome: | The proposed model outperforms existing models in qualitative analyses compared to Korean monolingual models. |
TopXGen: Topic-Diverse Parallel Data Generation for Low-Resource Machine Translation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | In-context learning and similarity search have been shown to improve LLMs' performance in machine translation, but they lag behind when dealing with low-resource languages. |
| Approach: | They propose a method that uses an LLM to generate topic-specific target-side data in the LRL. |
| Outcome: | The proposed approach boosts LLM translation performance during in-context learning and fine-tuning. |
When Your Cousin Has the Right Connections: Unsupervised Bilingual Lexicon Induction for Related Data-Imbalanced Languages (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for unsupervised bilingual lexicon induction depend on good quality static or contextual embeddings for both languages. |
| Approach: | They propose a method for unsupervised bilingual lexicon induction between a related LRL and a high-resource language that only requires inference on a masked language model of the HRL. |
| Outcome: | The proposed method performs well on low-resource languages with 5M tokens against Hindi . it is compared with existing methods on (mid-resourced) Marathi and Nepali . |