| Challenge: | Prior work in multilingual and cross-lingual speech recognition has been limited to a subset of the world's most-spoken languages. |
| Approach: | They propose to use phonemes and phonemes as pretraining objectives to encourage language-independent representations. |
| Outcome: | The proposed model is able to learn language-independent representations of speech using multilingual training. |
Similar Papers
A Survey of Multilingual Models for Automatic Speech Recognition (2022.lrec-1)
Copied to clipboard
| Challenge: | Automatic Speech Recognition (ASR) systems have achieved human-like performance for a few languages, but the majority of the world’s languages do not have usable systems due to the lack of large speech datasets to train these models. |
| Approach: | They propose to use unlabeled speech data to build multilingual ASR models that can be used for improved performance on low-resource languages. |
| Outcome: | The proposed models can be used to improve performance on low-resource languages by using unlabeled speech data. |
Common Voice: A Massively-Multilingual Speech Corpus (2020.lrec-1)
Copied to clipboard
Rosana Ardila, Megan Branson, Kelly Davis, Michael Kohler, Josh Meyer, Michael Henretty, Reuben Morais, Lindsay Saunders, Francis Tyers, Gregor Weber
| Challenge: | Common Voice is a massively-multilingual collection of transcribed speech intended for speech technology research and development. |
| Approach: | They propose to use Mozilla’s DeepSpeech Speech-to-Text toolkit to perform multilingual automatic speech recognition experiments. |
| Outcome: | The proposed corpus is the largest in the public domain for speech recognition, both in terms of hours and languages. |
Multi-Staged Cross-Lingual Acoustic Model Adaption for Robust Speech Recognition in Real-World Applications - A Case Study on German Oral History Interviews (2020.lrec-1)
Copied to clipboard
| Challenge: | Current automatic speech recognition systems show remarkable performance when adequate data is used for training. |
| Approach: | They propose to perform a robust acoustic model adaption to a target domain in a cross-lingual manner. |
| Outcome: | The proposed approach reduces word error rate by more than 30% on German oral history interviews compared to a model trained from scratch on the target domain and 6-7% on same-language out-of-domain training data. |
Speech Translation and the End-to-End Promise: Taking Stock of Where We Are (2020.acl-main)
Copied to clipboard
| Challenge: | Until recently, the only feasible approach to translating acoustic speech signals into text was the cascaded approach. |
| Approach: | They propose a classification of the main challenges of traditional approaches to speech translation . they argue that end-to-end models fall short due to compromises made to address data scarcity . |
| Outcome: | This paper provides a brief survey of the main challenges of traditional approaches in speech translation . it reveals that many end-to-end models fail due to compromises made to address data scarcity. |
Language Lives in Sparse Dimensions: Toward Interpretable and Efficient Multilingual Control for Large Language Models (2026.eacl-long)
Copied to clipboard
| Challenge: | Prior studies show that large language models map multilingual content into English-aligned representations at intermediate layers before projecting them back into target-language token spaces in the later layers. |
| Approach: | They propose a method to identify and manipulate dimensions that are sparse and sparsity-based . they propose to use as few as 50 sentences of either parallel or monolingual data to manipulate these dimensions . |
| Outcome: | Experiments on a multilingual generation control task show the interpretability of these dimensions. |
The Interpreter Understands Your Meaning: End-to-end Spoken Language Understanding Aided by Speech Translation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Modern artificial intelligence is characterized by large pretrained language models with strong language capabilities to be adapted to various downstream tasks. |
| Approach: | They propose to use the task of speech translation (ST) to pretrain speech models for end-to-end SLU on intra- and cross-lingual scenarios. |
| Outcome: | The proposed model achieves higher performance over baselines on monolingual and multilingual intent classification as well as spoken question answering using SLURP, MINDS-14, and NMSQA benchmarks. |
Centurio: On Drivers of Multilingual Ability of Large Vision-Language Model (2025.acl-long)
Copied to clipboard
Gregor Geigle, Florian Schneider, Carolin Holtermann, Chris Biemann, Radu Timofte, Anne Lauscher, Goran Glavaš
| Challenge: | Existing models for large vision-language tasks are trained on English data, which makes them struggle to understand non-English input and fail to generate output in the desired target language. |
| Approach: | They conduct multi-stage experiments on 13 vision-language tasks and 43 languages . they find that one can include as many as 100 training languages simultaneously with as little as 25-50% of non-English data . |
| Outcome: | The proposed model outperforms existing models in 14 tasks and 56 languages. |
Cross-Lingual Event Detection via Optimized Adversarial Training (2022.naacl-main)
Copied to clipboard
| Challenge: | Recent work in this area has harnessed the language-invariant qualities of pre-trained Multi-lingual Language Models. |
| Approach: | They propose to use adversarial language adaptation to train a model to detect events in a target language. |
| Outcome: | The proposed model achieves state-of-the-art on 8 different language pairs, using 4 languages from unrelated families. |
Expanding Pretrained Models to Thousands More Languages via Lexicon-based Adaptation (2022.acl-long)
Copied to clipboard
| Challenge: | Recent studies have found that the performance of multilingual pretrained models is highly dependent on the availability of monolingual or parallel text in a target language. |
| Approach: | They propose to use bilingual lexicons to synthesize textual or labeled data and combine it with monolingual or parallel text when available. |
| Outcome: | The proposed methods improve performance for 19 under-represented languages with and without extra monolingual text. |
Taxi1500: A Dataset for Multilingual Text Classification in 1500 Languages (2025.naacl-short)
Copied to clipboard
| Challenge: | a large-scale text classification dataset encompassing 1504 languages is needed to address this gap . low-resource languages are often overlooked due to the scarcity of evaluation datasets. |
| Approach: | They propose to use translations of the Bible to construct a large-scale text classification dataset that covers 1504 languages and annotate them using crowdsourcing. |
| Outcome: | The proposed dataset covers 1504 languages and is available to the public. |