Code-Switching Information Retrieval: Benchmarks, Analysis, and the Limits of Current Retrievers (2026.findings-acl)
Copied to clipboard
Qingcheng Zeng, Yuheng Lu, Zeqi Zhou, Heli Qi, Puxuan Yu, Fuheng Zhao, Hitomi Yanaka, Weihao Xuan, Naoto Yokoya
| Challenge: | a new study examines the performance of code-switching IR in monolingual contexts . code-witching is a pervasive linguistic phenomenon in global communication . |
| Approach: | They propose a benchmark to evaluate code-switching IR in monolingual contexts . they propose CS-MTEB, which measures performance declines of up to 27% . |
| Outcome: | The proposed benchmark shows that code-switching performance is degraded by 27% . the proposed benchmark is based on a dataset of mixed-language queries . |
Similar Papers
Lost in the Mix: Evaluating LLM Understanding of Code-Switched Text (2026.acl-long)
Copied to clipboard
| Challenge: | Code-switching (CSW) is widespread in multilingual communities and increasingly prevalent in online content. |
| Approach: | They propose a pipeline for producing linguistically grounded CSW variants of established benchmarks across five typologically diverse languages. |
| Outcome: | The proposed model sets show that inserting non-English tokens into English reduces accuracy on comprehension and reasoning benchmarks, whereas embedding English into non- English contexts often improves it. |
The Decades Progress on Code-Switching Research in NLP: A Systematic Survey on Trends and Challenges (2023.findings-acl)
Copied to clipboard
| Challenge: | Code-Switching is a common phenomenon in written text and conversation . it is not so common to observe code-switching in spoken language and not in written language . |
| Approach: | They present a systematic survey on code-switching research in natural language processing to understand the progress of the past decades and conceptualize the challenges and tasks on the topic. |
| Outcome: | The proposed model combines linguistic theories and machine learning techniques to understand the code-switching phenomenon. |
Beyond Monolingual Assumptions: A Survey on Code-Switched NLP in the Era of Large Language Models across Modalities (2026.acl-long)
Copied to clipboard
| Challenge: | Amidst the rapid advances of large language models, most LLMs struggle with mixed-language inputs, limited Code-switching datasets, and evaluation biases. |
| Approach: | They propose a roadmap for inclusive datasets, fair evaluation, and linguistically grounded models to achieve truly multilingual intelligence. |
| Outcome: | The proposed frameworks are based on 327 studies spanning five research areas, 15+ NLP tasks, 30+ datasets, and 80+ languages. |
Code-Mixed Probes Show How Pre-Trained Models Generalise on Code-Switched Text (2024.lrec-main)
Copied to clipboard
| Challenge: | Code-switching is a prevalent linguistic phenomenon in which multilingual individuals seamlessly alternate between languages. |
| Approach: | They propose to use pre-trained language models to generalise to code-switched text . they use a dataset of well-formed naturalistic code-witched texts and parallel translations into the source languages to examine their results. |
| Outcome: | The proposed model generalises to code-switched text, shedding light on their ability to generalise representations to CS corpora. |
Minimal Pair-Based Evaluation of Code-Switching (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods do not have wide language coverage, fail to account for the diverse range of CS phenomena, or do not scale. |
| Approach: | They propose to use minimal pairs of CS to estimate the extent to which large language models (LLMs) use code-switching in the same way as bilinguals. |
| Outcome: | The proposed model assigns higher probability to the naturally occurring CS sentence than to the variant for every language pair. |
Evaluating Code-Switching Translation with Large Language Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have shown they can match or surpass finetuned models on many natural language processing tasks. |
| Approach: | They propose to use in-context learning and pivot translation to improve code-switching translation. |
| Outcome: | The proposed models show strong ability for cross-lingual understanding in a code-switching setting. |
How effective is machine translation on low-resource code-switching? A case study comparing human and automatic metrics (2023.findings-acl)
Copied to clipboard
| Challenge: | Specifically, we compare the performance of three MT systems in terms of their ability to translate monolingual Vietnamese, a low-resource language, and Vietnamese-English CSW respectively. |
| Approach: | They compare the performance of three machine translation systems in the context of machine translation (MT) they find that state-of-the-art neural translation systems achieve higher scores on automatic metrics when processing CSW input . |
| Outcome: | The proposed system can translate monolingual Vietnamese, a low-resource language, and Vietnamese-English CSW respectively. |
Code-Switching and Syntax: A Large-Scale Experiment (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing theories of code-switching (CS) have been refuted in subsequent investigations. |
| Approach: | They propose to use syntactic information to predict where bilinguals switch languages . they find that syntax alone is sufficient for an automatic system to distinguish between sentences in minimal pairs of CS, to the same degree as bilingual humans. |
| Outcome: | The proposed model can explain why bilinguals switch languages more often than in others, but there is no large-scale, multi-language, cross-phenomena experiment that tests this claim. |
Boosting Zero-shot Cross-lingual Retrieval by Training on Artificially Code-Switched Data (2023.findings-acl)
Copied to clipboard
| Challenge: | Using zero-shot rankers, cross-lingual IR models are limited by their language coverage. |
| Approach: | They propose to train ranking models on artificially code-switched data instead of using a dictionary. |
| Outcome: | The proposed approach is robust towards the ratio of code-switched tokens and extends to unseen languages. |
Code-Switching Red-Teaming: LLM Evaluation for Safety and Multilingual Understanding (2025.acl-long)
Copied to clipboard
| Challenge: | Recent large language models (LLMs) are inherently multilingual agents . concerns regarding their safety have emerged . |
| Approach: | They propose a framework to synthesize red-teaming queries and investigate their safety . they demonstrate that the framework outperforms existing red- teaming techniques . |
| Outcome: | The proposed framework outperforms existing red-teaming techniques in the safety domain . it generates code-switching attack prompts in monolingual data . |