WER-BERT: Automatic WER Estimation with BERT in a Balanced Ordinal Classification Paradigm (2021.eacl-main)
Copied to clipboard
| Challenge: | Automatic Speech Recognition (ASR) systems are evaluated using Word Error Rate (WER) a higher WER means a lower percentage of errors between the ground truth and the transcription of the system. |
| Approach: | They propose a new balanced paradigm for automatic Word Error Rate estimation using a Librispeech dataset and a Google Cloud's Speech-to-Text API. |
| Outcome: | The proposed approach is more effective than regression in a classification setting, but suffers from heavy class imbalance. |
Similar Papers
Word Error Rate Estimation for Speech Recognition: e-WER (P18-2)
Copied to clipboard
| Challenge: | Automatic speech recognition (ASR) systems require manual transcription of test data to compute the word error rate (WER). |
| Approach: | They propose an approach to estimate word error rate (e-WER) that does not require a gold-standard transcription of the test set. |
| Outcome: | The proposed approach achieves 16.9% WER root mean squared error across 1,400 sentences. |
Automatic Speech Recognition System-Independent Word Error Rate Estimation (2024.lrec-main)
Copied to clipboard
| Challenge: | Word error rate (WER) is a metric used to evaluate the quality of transcriptions produced by Automatic Speech Recognition systems. |
| Approach: | They propose a hypothesis generation method for ASR system-dependent WER estimation . they use phonetically similar or linguistically more likely alternative words to generate hypotheses . |
| Outcome: | The proposed method outperforms baseline estimators on in-domain data and out-of-domain on Switchboard and CALLHOME. |
Using Automatic Speech Recognition in Spoken Corpus Curation (2020.lrec-1)
Copied to clipboard
| Challenge: | Automatic Speech Recognition (ASR) is a new way to make audio-visual data accessible. |
| Approach: | They propose to use automatic speech recognition (ASR) to make audio-visual data accessible by systematic queries. |
| Outcome: | The proposed system has higher recognition scores for the north of Germany vs. lower scores for south of the country. |
CEASR: A Corpus for Evaluating Automatic Speech Recognition (2020.lrec-1)
Copied to clipboard
Malgorzata Anna Ulasik, Manuela Hürlimann, Fabian Germann, Esin Gedik, Fernando Benites, Mark Cieliebak
| Challenge: | Automatic Speech Recognition (ASR) systems are increasingly needed for research and practical applications. |
| Approach: | They propose to use public speech corpora to evaluate the quality of automatic speech recognition (ASR) they calculate an average Word Error Rate (WER) per corpus, per system and per corpor-system pair . |
| Outcome: | The proposed corpus evaluates the quality of automatic speech recognition systems using public speech corpora and transcripts generated by state-of-the-art systems. |
The Influence of Automatic Speech Recognition on Linguistic Features and Automatic Alzheimer’s Disease Detection from Spontaneous Speech (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing biomarkers for AD diagnosis can only be applied to relatively small sample sizes due to limited availability, excessive costs and invasive nature. |
| Approach: | They compare automatic speech recognition systems in terms of Word Error Rate (WER) using a publicly available benchmark dataset of speech recordings of AD patients and controls. |
| Outcome: | The proposed method improves classification performance by replacing manual transcriptions with ASR output. |
A Benchmark of French ASR Systems Based on Error Severity (2025.coling-main)
Copied to clipboard
| Challenge: | Automatic Speech Recognition (ASR) transcription errors are often assessed using metrics that compare them with a reference transcription. |
| Approach: | They propose to categorize transcription errors into four levels of severity based on objective linguistic criteria, contextual patterns, and the use of content words as the unit of analysis. |
| Outcome: | The proposed evaluation categorizes errors into four levels of severity based on objective linguistic criteria, contextual patterns, and the use of content words as the unit of analysis. |
An Effective Automated Speaking Assessment Approach to Mitigating Data Scarcity and Imbalanced Distribution (2024.findings-naacl)
Copied to clipboard
| Challenge: | Automated speaking assessment (ASA) typically involves automatic speech recognition (ASR) and hand-crafted feature extraction from the transcript of a learner’s speech. |
| Approach: | They propose to use metric-based classification and loss re-weighting to model the impact of different SSL-based embedding features on the CEFR score. |
| Outcome: | The proposed model outperforms baselines on the ICNALE benchmark dataset, achieving a significant improvement of more than 10% in CEFR prediction accuracy. |
Advocating Character Error Rate for Multilingual ASR Evaluation (2025.findings-naacl)
Copied to clipboard
| Challenge: | Word error rate (WER) has been used for automatic speech recognition (ASR) evaluations for English datasets for many years. |
| Approach: | They propose to use the character error rate as the primary metric in multilingual ASR evaluation to account for morphologically complex languages. |
| Outcome: | The character error rate (CER) is the primary evaluation metric in multilingual ASR evaluation. |
On the Robust Approximation of ASR Metrics (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for estimating speech recognition metrics depend on ground truth labels. |
| Approach: | They propose a label-free approach to approximating ASR performance metrics . they embed multimodal embeddings in a unified space for speech and transcription representations . |
| Outcome: | The proposed method outperforms baseline models on speech recognition benchmarks by 50%. |
WESR: A Benchmark and Strong Baseline for Word-level Event-Speech Recognition (2026.findings-acl)
Copied to clipboard
Chenchen Yang, Kexin Huang, Liwei Fan, Qian Tu, Botian Jiang, Dong Zhang, Linqi Yin, Shimin Li, Zhaoye Fei, Qinyuan Cheng, Xipeng Qiu
| Challenge: | aaron carroll: the precise localization of non-verbal vocal events remains a critical yet under-explored challenge. carroll says current methods suffer from insufficient task definitions with limited category coverage. carrol: knowing exactly where an event occurred is not enough; knowing exactly what it happened is. |
| Approach: | They propose a taxonomy of 21 vocal events with a new categorization into discrete versus continuous types. |
| Outcome: | The proposed model disentangles ASR errors from event detection while maintaining ASR quality. |