Code-Switching Curriculum Learning for Multilingual Transfer in LLMs (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) exhibit near human-level performance in various tasks, but performance drops after a handful of high-resource languages due to the imbalance in pre-training data. |
| Approach: | They propose a code-switching curriculum learning model to enhance cross-lingual transfer for LLMs by progressively training models with a curriculum consisting of token-level code-changing, sentence-level codeswitching, and monolingual corpora. |
| Outcome: | The proposed model improves language transfer to Korean, with significant gains in Japanese and Indonesian . the proposed model mitigates spurious correlations between language resources and safety alignment . |
Similar Papers
Investigating and Scaling up Code-Switching for Multilingual Language Model Pre-Training (2025.findings-acl)
Copied to clipboard
Zhijun Wang, Jiahuan Li, Hao Zhou, Rongxiang Weng, Jingang Wang, Xin Huang, Xue Han, Junlan Feng, Chao Deng, Shujian Huang
| Challenge: | Large language models (LLMs) exhibit remarkable multilingual capabilities despite the extreme language imbalance in the pre-training data. |
| Approach: | They investigate the existence of code-switching in the pre-training corpus and categorize it into four types within two quadrants. |
| Outcome: | The proposed approach improves performance across benchmarks and representation space. |
Can Code-Switched Texts Activate a Knowledge Switch in LLMs? A Case Study on English-Korean Code-Switching (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Recent large language models (LLMs) demonstrate multilingual abilities, yet they are English-centric due to dominance of English in training corpora. |
| Approach: | They propose to use a synthetic English-korean CS question-answering dataset to investigate this potential. |
| Outcome: | The proposed model can activate, identify and leverage knowledge for reasoning in low-resource languages. |
MIGRATE: Cross-Lingual Adaptation of Domain-Specific LLMs through Code-Switching and Embedding Transfer (2025.coling-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have advanced in many fields, but focus on English-centric models requires extensive data. |
| Approach: | They propose a method that leverages open-source static embedding models and up to 3 million tokens of code-switching data to facilitate the seamless transfer of embeddables to target languages. |
| Outcome: | The proposed method outperforms baseline and existing cross-lingual transfer methods in target languages. |
Lost in the Mix: Evaluating LLM Understanding of Code-Switched Text (2026.acl-long)
Copied to clipboard
| Challenge: | Code-switching (CSW) is widespread in multilingual communities and increasingly prevalent in online content. |
| Approach: | They propose a pipeline for producing linguistically grounded CSW variants of established benchmarks across five typologically diverse languages. |
| Outcome: | The proposed model sets show that inserting non-English tokens into English reduces accuracy on comprehension and reasoning benchmarks, whereas embedding English into non- English contexts often improves it. |
Evaluating Code-Switching Translation with Large Language Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have shown they can match or surpass finetuned models on many natural language processing tasks. |
| Approach: | They propose to use in-context learning and pivot translation to improve code-switching translation. |
| Outcome: | The proposed models show strong ability for cross-lingual understanding in a code-switching setting. |
Code-Switching Red-Teaming: LLM Evaluation for Safety and Multilingual Understanding (2025.acl-long)
Copied to clipboard
| Challenge: | Recent large language models (LLMs) are inherently multilingual agents . concerns regarding their safety have emerged . |
| Approach: | They propose a framework to synthesize red-teaming queries and investigate their safety . they demonstrate that the framework outperforms existing red- teaming techniques . |
| Outcome: | The proposed framework outperforms existing red-teaming techniques in the safety domain . it generates code-switching attack prompts in monolingual data . |
Beyond Monolingual Assumptions: A Survey on Code-Switched NLP in the Era of Large Language Models across Modalities (2026.acl-long)
Copied to clipboard
| Challenge: | Amidst the rapid advances of large language models, most LLMs struggle with mixed-language inputs, limited Code-switching datasets, and evaluation biases. |
| Approach: | They propose a roadmap for inclusive datasets, fair evaluation, and linguistically grounded models to achieve truly multilingual intelligence. |
| Outcome: | The proposed frameworks are based on 327 studies spanning five research areas, 15+ NLP tasks, 30+ datasets, and 80+ languages. |
Toward the Limitation of Code-Switching in Cross-Lingual Transfer (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have shown the success of multilingual pretrained models for cross-lingual knowledge transfer. |
| Approach: | They propose to make code-switched sentences replace tokens from multiple languages so they are grammatically consistent . they also consider the similarity between context and the switched tokens to ensure that the newly substituted sentences are grammatically consistent - a limitation that could affect inference . |
| Outcome: | The proposed method outperforms the mBERT and original code-switching method on cross-lingual POS and Named-Entity-Recognition tasks on 30+ languages. |
Making LLMs Better Many-to-Many Speech-to-Text Translators with Curriculum Learning (2025.acl-long)
Copied to clipboard
Yexing Du, Youcheng Pan, Ziyang Ma, Bo Yang, Yifan Yang, Keqi Deng, Xie Chen, Yang Xiang, Ming Liu, Bing Qin
| Challenge: | Existing studies on English-centric translation tasks have focused on multimodal large language models, but the exploration of many-to-many translation is limited by the scarcity of parallel data. |
| Approach: | They propose a three-stage curriculum learning strategy that leverages the machine translation capabilities of large language models and adapts them to S2TT tasks. |
| Outcome: | The proposed strategy achieves state-of-the-art average performance in 1514 language pairs, requiring fewer than 10 hours of speech data per language to achieve competitive results. |
Meta-Transfer Learning for Code-Switched Speech Recognition (2020.acl-main)
Copied to clipboard
| Challenge: | Increasing number of people in the world today speak a mixed-language as a result of being multilingual. |
| Approach: | They propose a method to transfer learn on a code-switched speech recognition system by extracting information from high-resource monolingual datasets. |
| Outcome: | The proposed model outperforms baselines on speech recognition and language modeling tasks and is faster to converge. |