Cross-Lingual Transfer from Related Languages: Treating Low-Resource Maltese as Multilingual Code-Switching (2024.eacl-long)
Copied to clipboard
| Challenge: | Multilingual models exhibit impressive cross-lingual transfer capabilities on unseen languages, but performance is impacted when there is a script disparity with the languages used in the model’s pre-training data. |
| Approach: | They propose a novel method to align a resource-rich language's script with a target language and train a classifier that can make informed decisions regarding the appropriate processing of each token. |
| Outcome: | The proposed model can be used to transfer a language's scripts across multiple languages, but it is suboptimal for mixed languages, where only a subset benefits while the rest is impeded. |
Similar Papers
Breaking the Script Barrier in Multilingual Pre-Trained Language Models with Transliteration-Based Post-Training Alignment (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent mPLMs have shown impressive performance on crosslingual transfer tasks . however, the performance is often hindered when a lowresource target language is written in a different script than the high-resource source language. |
| Approach: | They propose a transliteration-based method to improve cross-lingual alignment between languages using diverse scripts. |
| Outcome: | The proposed method outperforms the original model on Englishcentric transfer tasks up to 50%. |
Code-Switching and Back-Transliteration Using a Bilingual Model (2024.findings-eacl)
Copied to clipboard
| Challenge: | a wealth of Judeo-Arabic texts is available on the internet, but most speakers of Arabic are unfamiliar with the Hebrew script. |
| Approach: | They propose two machine-learning models for transliterating Judeo-Arabic into Arabic . they also propose a model for identifying non-Arabian words, predominantly Hebrew and Aramaic . |
| Outcome: | The proposed models outperform previous solutions for transliterating Judeo-Arabic into Arabic and identifying non-Arabian words, mainly Hebrew and Aramaic. |
Data Augmentation for Maltese NLP using Transliterated and Machine Translated Arabic Data (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Maltese is a Semitic language that has evolved under extensive influence from Romance and Germanic languages, particularly Italian and English. |
| Approach: | They investigate whether Arabic-language resources can support Maltese natural language processing . they introduce transliteration schemes and machine translation approaches to align Arabic text with Maltesen . |
| Outcome: | The proposed techniques can significantly improve Maltese natural language processing tasks. |
T3L: Translate-and-Test Transfer Learning for Cross-Lingual Text Classification (2023.tacl-1)
Copied to clipboard
| Challenge: | Existing approaches to cross-lingual text classification leverage text classifiers trained in a high-resource language to perform text classification in other languages with no or minimal fine-tuning. |
| Approach: | They propose to combine a neural machine translator and a text classifier trained in a high-resource language to perform text classification in other languages with no or minimal fine-tuning. |
| Outcome: | The proposed approach significantly improves over a baseline approach. |
Enhancing Cross-Lingual Transfer through Reversible Transliteration: A Huffman-Based Approach for Low-Resource Languages (2025.acl-long)
Copied to clipboard
| Challenge: | Large language models demonstrate cross-lingual transfer capabilities, but these capabilities often fail to extend to low-resource languages, especially those utilizing non-Latin scripts. |
| Approach: | They propose to combine character transliteration with Huffman coding to create a complete transliterations framework that can be extended to other low-resource languages. |
| Outcome: | The proposed framework reduces storage requirements and improves accuracy and accuracy across multiple downstream tasks while maintaining performance on high-resource languages. |
Multilingual unsupervised sequence segmentation transfers to extremely low-resource languages (2022.acl-long)
Copied to clipboard
| Challenge: | Unsupervised sequence segmentation is a key component of low-resource languages where there is little or no gold-standard data on which to train supervised models. |
| Approach: | They propose to pre-train a Masked Segmental Language Model multilingually to achieve unsupervised segmentation performance in extremely low-resource languages. |
| Outcome: | The proposed model outperforms a monolingual model and a pre-trained model on Quechua in 6/10 settings. |
Unifying Cross-Lingual Transfer across Scenarios of Resource Scarcity (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to deal with resource scarcity have not been developed to deal effectively with the problem. |
| Approach: | They propose to use a set of tools to harness data from one or more high-resource "source" languages to compensate for a shortage of data in low-resourced "target" languages. |
| Outcome: | The proposed technique can be easily adapted to unseen languages, extending the range of the proposed technique and translation-based transfer more broadly. |
Unknown Script: Impact of Script on Cross-Lingual Transfer (2024.naacl-srw)
Copied to clipboard
| Challenge: | Existing models for high-resource languages are not available for all languages, and the vast majority of the world's languages are excluded from these models. |
| Approach: | They propose to use pre-trained models to analyze the effect of the target language and its script on cross-lingual transfer. |
| Outcome: | The proposed model is based on six models pre-trained on NER and POS tasks in the original script and romanized version. |
Scripts Through Time: A Survey of the Evolving Role of Transliteration in NLP (2026.findings-acl)
Copied to clipboard
| Challenge: | Cross-lingual transfer is often hindered by the "script barrier" where differences in writing systems inhibit transfer learning . transliteration is a powerful technique to bridge this gap by increasing lexical overlap . authors present a taxonomy of key motivations to utilize transliterations in language models . |
| Approach: | They propose a taxonomy of key motivations to utilize transliterations in NLP . they analyze the evolution and effectiveness of these methods and discuss trade-offs . |
| Outcome: | The proposed transliteration technique is effective in cross-lingual NLP, the authors argue . the proposed translliteration method is a powerful tool to overcome the "script barrier" |
The Role of Mixed-Language Documents for Multilingual Large Language Model Pretraining (2026.acl-long)
Copied to clipboard
Jiandong Shao, Raphael Tang, Crystina Zhang, Karin Sevegnani, Pontus Stenetorp, Jianfei Yang, Yao Lu
| Challenge: | Existing research suggests that multilingual large language models can achieve impressive cross-lingual understanding despite largely monolingual pretraining. |
| Approach: | They compare a monolingual-only corpus with a standard web corpus that removes all multilingual documents and then retrain the models from scratch under controlled conditions. |
| Outcome: | The results show that removing bilingual data causes translation performance to drop 56% in BLEU, whereas code-switching contributes minimally. |