Happiness is Sharing a Vocabulary: A Study of Transliteration Methods (2026.eacl-long)
Copied to clipboard
| Challenge: | a key problem in multilingual NLP is script barrier, which makes it difficult to share knowledge between languages . a new study shows that transliteration can be useful for languages using non-Latin scripts . |
| Approach: | They propose to use romanization, phonemic transcription, and substitution ciphers to evaluate models . romanization outperforms other input types in 7 out of 8 evaluation settings . |
| Outcome: | The proposed approach outperforms other input types on three tasks and is the most effective . romanization outperformed other input type in 7 out of 8 evaluation settings . |
Similar Papers
Scripts Through Time: A Survey of the Evolving Role of Transliteration in NLP (2026.findings-acl)
Copied to clipboard
| Challenge: | Cross-lingual transfer is often hindered by the "script barrier" where differences in writing systems inhibit transfer learning . transliteration is a powerful technique to bridge this gap by increasing lexical overlap . authors present a taxonomy of key motivations to utilize transliterations in language models . |
| Approach: | They propose a taxonomy of key motivations to utilize transliterations in NLP . they analyze the evolution and effectiveness of these methods and discuss trade-offs . |
| Outcome: | The proposed transliteration technique is effective in cross-lingual NLP, the authors argue . the proposed translliteration method is a powerful tool to overcome the "script barrier" |
Does Transliteration Help Multilingual Language Modeling? (2023.findings-eacl)
Copied to clipboard
| Challenge: | Script diversity presents a challenge to Multilingual Language Models by reducing lexical overlap . Script diversification can be used to improve performance of MLLMs by transliterating closely related languages to a common script. |
| Approach: | They empirically measure the effect of transliteration on MLLMs by focusing on Indic languages . they find that transliterations benefit low-resource languages without negatively affecting high-resourced ones . |
| Outcome: | The proposed transliteration-based model learns sentences that are more similar across languages. |
One Script Instead of Hundreds? On Pretraining Romanized Encoder Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | a recent study has focused on setups that favor romanization for cross-lingual transfer . a fidelity-based approach is needed to improve performance for high-resource languages . |
| Approach: | They propose to pretrain LMs from scratch on romanized and original texts for six languages . they find that romanization improves encoding efficiency for segmental scripts at a negligible cost . |
| Outcome: | The proposed method reduces the loss of script-specific information and dilution of language-specific representations from increased subword overlap. |
How Transliterations Improve Crosslingual Alignment (2025.coling-main)
Copied to clipboard
Yihong Liu, Mingyang Wang, Amir Hossein Kargaran, Ayyoob ImaniGooghari, Orgest Xhelili, Haotian Ye, Chunlan Ma, François Yvon, Hinrich Schütze
| Challenge: | Recent studies show that post-aligning multilingual pretrained language models improve crosslingual alignment, but it is unclear how and why this is achieved. |
| Approach: | They propose to explicitly evaluate crosslingual alignment by adding transliterations to models using original and transliterated data. |
| Outcome: | The proposed approach improves crosslingual alignment even for random sentences. |
Part-of-Speech Tagging for Code-Switched, Transliterated Texts without Explicit Language Identification (D18-1)
Copied to clipboard
| Challenge: | Code-switching is a challenge for NLP due to the lack of representative data for training models. |
| Approach: | They propose a model that is trained exclusively on monolingual resources but can be applied to unseen code-switched text at inference time. |
| Outcome: | The proposed model outperforms standard models on Hindi-English part-of-speech tagging and on unannotated code-switched text with alternate scripts. |
Beyond Shared Vocabulary: Increasing Representational Word Similarities across Languages for Multilingual Machine Translation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Using a shared vocabulary is common practice in multilingual machine translation . however, when words overlap is small, e.g., using different writing systems, knowledge transfer is inhibited . |
| Approach: | They propose a re-parameterized method for building word embeddings using word equivalence classes and graph networks to fuse word embeds across languages. |
| Outcome: | The proposed method achieves evident BLEU improvements on high- and low-resource MNMT scenarios. |
Breaking the Script Barrier in Multilingual Pre-Trained Language Models with Transliteration-Based Post-Training Alignment (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent mPLMs have shown impressive performance on crosslingual transfer tasks . however, the performance is often hindered when a lowresource target language is written in a different script than the high-resource source language. |
| Approach: | They propose a transliteration-based method to improve cross-lingual alignment between languages using diverse scripts. |
| Outcome: | The proposed method outperforms the original model on Englishcentric transfer tasks up to 50%. |
Tokenization Impacts Multilingual Language Modeling: Assessing Vocabulary Allocation and Overlap Across Languages (2023.findings-acl)
Copied to clipboard
| Challenge: | Multilingual language models perform surprisingly well in a variety of NLP tasks for diverse languages. |
| Approach: | They propose to evaluate the quality of lexical representation and vocabulary overlap observed in sub-word tokenizers. |
| Outcome: | The proposed criteria show that the overlap of vocabulary across languages can be detrimental to certain downstream tasks. |
False Friends Are Not Foes: Investigating Vocabulary Overlap in Multilingual Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Prior work has shown that token overlap facilitates cross-lingual transfer or introduces interference between languages? |
| Approach: | They devised a controlled experiment where they train bilingual autoregressive models on multiple language pairs under systematically varied vocabulary overlap settings. |
| Outcome: | The proposed model outperforms models with disjointed vocabularies on XNLI and XQuAD and shows that token overlap is beneficial for multilingual tokenizers. |
Where are we Still Split on Tokenization? (2024.findings-eacl)
Copied to clipboard
| Challenge: | Identifying tokens is a crucial first step for many tasks in Natural Language Processing (NLP) gold tokenization is often assumed, but some work on token-level tasks is more challenging. |
| Approach: | They propose an efficient method for tokenization with subword-based language models and evaluate it on 122 languages in 20 scripts. |
| Outcome: | The proposed method performs on par with the state-of-the-art on 122 languages in 20 scripts. |