RomanLens: The Role Of Latent Romanization In Multilinguality In LLMs (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) exhibit strong multilingual performance despite training on English-centric corpora. |
| Approach: | They propose to use Romanization as a potential bridge in multilingual processing . they propose to encode semantic concepts similarly across native and Romanized scripts . |
| Outcome: | The proposed model encodes semantic concepts across native and Romanized scripts, suggesting a shared underlying representation. |
Similar Papers
One Script Instead of Hundreds? On Pretraining Romanized Encoder Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | a recent study has focused on setups that favor romanization for cross-lingual transfer . a fidelity-based approach is needed to improve performance for high-resource languages . |
| Approach: | They propose to pretrain LMs from scratch on romanized and original texts for six languages . they find that romanization improves encoding efficiency for segmental scripts at a negligible cost . |
| Outcome: | The proposed method reduces the loss of script-specific information and dilution of language-specific representations from increased subword overlap. |
Language-Specific Neurons: The Key to Multilingual Capabilities in Large Language Models (2024.acl-long)
Copied to clipboard
Tianyi Tang, Wenyang Luo, Haoyang Huang, Dongdong Zhang, Xiaolei Wang, Xin Zhao, Furu Wei, Ji-Rong Wen
| Challenge: | Despite the impressive multilingual capabilities demonstrated by LLMs, the understanding of how these abilities develop and function remains nascent. |
| Approach: | They propose a novel detection method to pinpoint language-specific neurons within LLMs by selectively activating or deactivating these neurons. |
| Outcome: | The proposed method can “steer” the output language of LLMs by selectively activating or deactivating language-specific neurons. |
Converging to a Lingua Franca: Evolution of Linguistic Regions and Semantics Alignment in Multilingual Large Language Models (2025.coling-main)
Copied to clipboard
| Challenge: | Recent studies suggest that large language models can transfer skills learned in one language to others, but internal mechanisms behind this ability remain unclear. |
| Approach: | They find that LLMs map semantically identical inputs from different languages into a common semantic latent space that allows for consistent processing across languages. |
| Outcome: | The findings highlight the structural evolution of multilingual models during training and scaling up. |
Prompting with Phonemes: Enhancing LLMs’ Multilinguality for Non-Latin Script Languages (2025.naacl-long)
Copied to clipboard
Hoang H Nguyen, Khyati Mahajan, Vikas Yadav, Julian Salazar, Philip S. Yu, Masoud Hashemi, Rishabh Maheshwary
| Challenge: | Multilingual LLMs have achieved remarkable benchmark performance, but continue to underperform on non-Latin script languages. |
| Approach: | They propose to integrate phonemic transcriptions as complementary signals to induce script-invariant representations by integrating phonemic and orthographic transcriptions. |
| Outcome: | The proposed approach improves performance for Latin and non-Latin script languages, with 12.6% performance improvement and 15.1% performance improvement compared to randomized ICL retrieval. |
Representational Isomorphism and Alignment of Multilingual Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing isomorphism of sentence representations can facilitate representational alignments in zero-shot and few-shot settings. |
| Approach: | They propose to apply a contrastive objective to LLMs with a small number of translation pairs to improve models' performance on Semantic Textual Similarity tasks. |
| Outcome: | The proposed representation-level approach significantly improves on Semantic Textual Similarity (STS) tasks across languages even without a monolingual objective. |
The Transfer Neurons Hypothesis: An Underlying Mechanism for Language Latent Space Transitions in Multilingual LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies suggest a processing framework for multilingual inputs in decoder-based LLMs. |
| Approach: | They propose a framework for multilingual inputs in decoder-based LLMs that enables transfer of representations between latent spaces and shared semantic latent space. |
| Outcome: | The proposed framework is validated by a new study on transfer neurons in multilingual LLMs. |
Can you map it to English? The Role of Cross-Lingual Alignment in the Multilingual Performance of LLMs (2026.eacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) can answer prompts in many languages despite being pre-trained mostly on English text. |
| Approach: | They propose a Discriminative Alignment Index to quantify instance-level alignment across 24 languages other than English and three distinct NLU tasks. |
| Outcome: | The proposed model can perform natural language understanding tasks in 24 languages other than English and three distinct NLU tasks. |
Hyperpolyglot LLMs: Cross-Lingual Interpretability in Token Embeddings (2023.emnlp-main)
Copied to clipboard
| Challenge: | XLMs can support cross-lingual transfer learning with little to no additional training data. |
| Approach: | They describe a mechanism for cross-lingual transfer learning by measuring the properties of the initial token embedding layer. |
| Outcome: | The proposed model can be used to support cross-lingual transfer learning . the initial token embedding layer is expressive and interpretable . |
Multilingual Machine Translation with Large Language Models: Empirical Results and Analysis (2024.findings-naacl)
Copied to clipboard
Wenhao Zhu, Hongyi Liu, Qingxiu Dong, Jingjing Xu, Shujian Huang, Lingpeng Kong, Jiajun Chen, Lei Li
| Challenge: | Existing studies show that large language models (LLMs) can handle multilingual machine translation (MMT) However, the multilingual translation ability of LLMs remains under-explored. |
| Approach: | They evaluate eight popular LLMs including ChatGPT and GPT-4 to determine their performance in multilingual machine translation. |
| Outcome: | The proposed model can generate moderate translation even on zero-resource languages and cross-lingual exemplars can provide better task guidance for low-resourced translation than exemplar in the same language pairs. |
Is Translation All You Need? A Study on Solving Multilingual Tasks with Large Language Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated multilingual capabilities, yet they are mostly English-centric due to the imbalanced training corpora. |
| Approach: | They extend the evaluation to real-world user queries and non-English-centric LLMs . they show that translation into English can boost LLM performance on NLP tasks . |
| Outcome: | The proposed evaluation extends to user queries and non-English-centric LLMs . it shows that translation into English can boost performance on NLP tasks, but not universally optimal . |