Evaluating Self-Supervised Speech Representations for Indigenous American Languages (2024.lrec-main)
Copied to clipboard
| Challenge: | a recent study focused on the use of self-supervised learning to learn speech representations for indigenous languages . aaron e. scott: the vast linguistic diversity represented by indigenous languages remains unexplored . by expanding the scope of language processing to include indigenous languages, we can foster linguistic inclusivity, he says . |
| Approach: | They benchmark the efficacy of large-scale self-supervised learning models on indigenous American languages. |
| Outcome: | The proposed model can generalize to real-world data, showing strong performance . evaluators found that the model performed better than monolingual models on indigenous languages . |
Similar Papers
Self-supervised Representation Learning for Speech Processing (2022.naacl-tutorials)
Copied to clipboard
Hung-yi Lee, Abdelrahman Mohamed, Shinji Watanabe, Tara Sainath, Karen Livescu, Shang-Wen Li, Shu-wen Yang, Katrin Kirchhoff
| Challenge: | Self-supervised representation learning (SSL) uses proxy supervised learning tasks to obtain training data from unlabeled corpora. |
| Approach: | They propose to survey the latest SSL techniques, tools, datasets, and performance achievement in speech processing to scale up current machine learning technologies. |
| Outcome: | The proposed tutorial is highly relevant to the special theme of ACL about language diversity. |
A Survey of Multilingual Models for Automatic Speech Recognition (2022.lrec-1)
Copied to clipboard
| Challenge: | Automatic Speech Recognition (ASR) systems have achieved human-like performance for a few languages, but the majority of the world’s languages do not have usable systems due to the lack of large speech datasets to train these models. |
| Approach: | They propose to use unlabeled speech data to build multilingual ASR models that can be used for improved performance on low-resource languages. |
| Outcome: | The proposed models can be used to improve performance on low-resource languages by using unlabeled speech data. |
An (unhelpful) guide to selecting the best ASR architecture for your under-resourced language (2023.acl-short)
Copied to clipboard
| Challenge: | English ASR now has word error rates comparable to that of human transcriptionists, but only for the handful of the world's 7000 languages with abundant training resources. |
| Approach: | They propose to use four of the most popular ASR toolkits to train ASR models for eleven languages with limited ASR training resources: eleven widely spoken languages of Africa, Asia, and South America, one endangered language of Central America, and three critically endangered languages of North America. |
| Outcome: | The proposed architecture outperforms four of the most popular ASR toolkits for eleven languages with limited training resources. |
An Exploration of Mamba for Speech Self-Supervised Models (2026.acl-long)
Copied to clipboard
Tzu-Quan Lin, Heng-Cheng Kuo, Tzu-Chieh Wei, Hsi-Chun Cheng, Chun Wei Chen, Hsien-Fu Hsiao, Yu Tsao, Hung-yi Lee
| Challenge: | Mamba-based SSL models are promising for long-sequence modeling, speech unit extraction, and speech self-supervised learning. |
| Approach: | They propose to use Mamba-based HuBERT models as an alternative to Transformer-based SSL architectures. |
| Outcome: | The proposed models outperform Transformer-based models in language modeling tasks while showing superior performance on streaming ASR. |
Fine-Tuning ASR models for Very Low-Resource Languages: A Study on Mvskoke (2024.acl-srw)
Copied to clipboard
| Challenge: | Recent advances in multilingual models for automatic speech recognition (ASR) have been able to achieve a high accuracy for languages with extremely limited resources. |
| Approach: | They examine the parameter efficiency of training an adapter for the Mvskoke language, an indigenous language of America. |
| Outcome: | The proposed model is parameter efficient and gives higher accuracy for a relatively small amount of data. |
Rethinking Semi-supervised Learning with Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Semi-supervised learning (SSL) is a popular setting to make use of unlabelled data . Currently, there are two popular approaches to make effective use of the unlabelled datasets . |
| Approach: | They compare semi-supervised learning (SSL) and task-adaptive pre-training (TAPT) they find TAPT is a stronger and more robust SSL learner, even when using just a few hundred unlabelled samples . |
| Outcome: | The proposed methods improve model performance across different NLP tasks and data sizes. |
Combining Weakly Supervised ML Techniques for Low-Resource NLU (2021.naacl-industry)
Copied to clipboard
| Challenge: | Recent advances in transfer learning have improved the performance of virtual assistants . however, meager training data is often a key bottleneck in creating voice-enabled applications . |
| Approach: | They propose to use unsupervised and semi-supervised techniques to improve NLU accuracy . they incorporate anonymized, unlabeled and automatically transcribed user utterances into training . |
| Outcome: | The proposed methods improve NLU accuracy in low-resource settings by integrating unsupervised and SSL techniques. |
Evaluating the Efficacy of Large Acoustic Model for Documenting Non-Orthographic Tribal Languages in India (2024.lrec-main)
Copied to clipboard
| Challenge: | Pre-trained Large Acoustic Models have been shown to improve performance in spoken languages . however, their potential for novel under-resourced languages is not fully known . |
| Approach: | They propose to use pre-trained Large Acoustic Models to document under-resourced languages . they use scripts from languages that hold a prominent presence in the geographical regions . |
| Outcome: | The proposed model can document under-resourced languages in the electronic domain . the model can be used to document languages with a written script . |
Unsupervised Cross-Lingual Representation Learning (P19-4)
Copied to clipboard
| Challenge: | a comprehensive survey of cutting-edge weakly-supervised and unsupervised cross-lingual word representations is presented . |
| Approach: | This tutorial provides a comprehensive survey of recent work on weakly-supervised and unsupervised cross-lingual word representations. |
| Outcome: | This tutorial provides a comprehensive survey of cutting-edge weakly-supervised and unsupervised word representations. |
Distant Supervision from Disparate Sources for Low-Resource Part-of-Speech Tagging (D18-1)
Copied to clipboard
| Challenge: | Low-resource languages lack manual annotated data to learn basic models such as part-of-speech (POS) taggers. |
| Approach: | They propose a cross-lingual neural part-of-speech tagger that learns from disparate sources of distant supervision in a uniform framework. |
| Outcome: | The proposed model scales to hundreds of low-resource languages without access to gold annotated data. |