PolyWER: A Holistic Evaluation Framework for Code-Switched Speech Recognition (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for measuring accuracy, such as Word Error Rate (WER), are too strict to address this challenge. |
| Approach: | They propose a framework for evaluating speech recognition systems to handle language-mixing by appending annotations to a publicly available Arabic-English code-switched dataset. |
| Outcome: | The proposed framework evaluates speech recognition systems against human judgement and a publicly available Arabic-English code-switched dataset. |
Similar Papers
Language Modeling for Code-Switching: Evaluation, Integration of Monolingual Data, and Discriminative Training (D19-1)
Copied to clipboard
| Challenge: | Code-switching (CS) is a linguistic phenomenon defined as "the alternation of two languages within a single discourse, sentence or constituent." |
| Approach: | They propose an ASR-motivated evaluation setup which is decoupled from an ASL system and the choice of vocabulary . they propose a discriminative training approach which works better than generative language modeling . |
| Outcome: | The proposed evaluation setup is better than generative language modeling, the authors show . the proposed setup is decoupled from an ASR system and the choice of vocabulary . |
DECM: Evaluating Bilingual ASR Performance on a Code-switching/mixing Benchmark (2024.lrec-main)
Copied to clipboard
| Challenge: | Code-switched (CSW) speech is a linguistic phenomenon that occurs when spoken utterances switch languages between sentences. |
| Approach: | They propose to use a dataset to evaluate German-English CSW speech . they show that the dataset includes splits with varying degrees of CSW . |
| Outcome: | The proposed dataset includes spontaneous speech from diverse domains, enabling realistic CSW evaluation in German-English. |
CoSSAT: Code-Switched Speech Annotation Tool (D19-59)
Copied to clipboard
| Challenge: | Code-switching is a phenomenon that occurs in multilingual societies where speakers who are fluent in two or more languages switch between these languages in the same conversation or utterance. |
| Approach: | They propose an interface which helps annotators transcribe code-switched speech faster, more easily and more accurately than a traditional interface. |
| Outcome: | The proposed interface can be used by 10 users to transcribe Hindi-English code-switched speech faster, easier and more accurately than a traditional interface. |
Improving Language Identification for Code-Switched Speech: The Pivotal Role of Accented English (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing models fail to identify English spoken with the accent of the matrix (dominant) language. |
| Approach: | They propose to fine tune existing LID models with accented English to improve code-switched LID . they use a metric that captures relative ranking of identified languages often overlooked by traditional metrics. |
| Outcome: | The proposed model can be fine tuned with small amounts of accented English without degrading performance on monolingual speech. |
CoSTA: Code-Switched Speech Translation using Aligned Speech-Text Interleaving (2025.coling-main)
Copied to clipboard
| Challenge: | More than half of the world's population is presumed to be bilingual . spoken translation of code-switched speech has been under-explored . |
| Approach: | They propose an end-to-end model architecture CoSTA that scaffolds on pretrained ASR and MT modules. |
| Outcome: | The proposed model outperforms existing models by 3.5 BLEU points in spoken translation of code-switched speech. |
HiKE: Hierarchical Evaluation Framework for Korean-English Code-Switching Speech Recognition (2026.findings-eacl)
Copied to clipboard
| Challenge: | Recent advances in automatic speech recognition (ASR) have pushed error rates below 5% on standard monolingual benchmarks. |
| Approach: | They propose a framework for the evaluation of multilingual ASR models using loanword labels and a hierarchical CS-level labeling scheme that allows for fine-tuning with synthetic CS data. |
| Outcome: | The proposed framework provides a means for the precise evaluation of multilingual ASR models and fosters research in the field. |
Part-of-Speech Tagging for Code-Switched, Transliterated Texts without Explicit Language Identification (D18-1)
Copied to clipboard
| Challenge: | Code-switching is a challenge for NLP due to the lack of representative data for training models. |
| Approach: | They propose a model that is trained exclusively on monolingual resources but can be applied to unseen code-switched text at inference time. |
| Outcome: | The proposed model outperforms standard models on Hindi-English part-of-speech tagging and on unannotated code-switched text with alternate scripts. |
Improving Code-switched ASR with Linguistic Information (2022.coling-1)
Copied to clipboard
| Challenge: | Existing studies on code-switching have been limited to the individual languages, but the results are promising. |
| Approach: | They propose to apply linguistic theories to generate more realistic code-switching text, which is needed for language modelling in ASR. |
| Outcome: | The proposed system improves 2% on English-Spanish code-switching . Equivalence Constraint theory and part-of-speech labelling are particularly helpful for text generation and bring 2% improvement to ASR performance. |
Beyond Monolingual Assumptions: A Survey on Code-Switched NLP in the Era of Large Language Models across Modalities (2026.acl-long)
Copied to clipboard
| Challenge: | Amidst the rapid advances of large language models, most LLMs struggle with mixed-language inputs, limited Code-switching datasets, and evaluation biases. |
| Approach: | They propose a roadmap for inclusive datasets, fair evaluation, and linguistically grounded models to achieve truly multilingual intelligence. |
| Outcome: | The proposed frameworks are based on 327 studies spanning five research areas, 15+ NLP tasks, 30+ datasets, and 80+ languages. |
LinCE: A Centralized Benchmark for Linguistic Code-switching Evaluation (2020.lrec-1)
Copied to clipboard
| Challenge: | Recent trends in NLP research have raised an interest in linguistic code-switching . however, many of these approaches are limited to a few language pairs and a specific domain . |
| Approach: | They propose a centralized benchmark for Linguistic Code-switching Evaluation that combines eleven corpora covering four different code-switch languages and four tasks. |
| Outcome: | The proposed benchmark provides a centralized benchmark and compares with other benchmarks in real-time. |