Nima Ebadi, Kellen Morgan, Adrian Tan, Billy Linares, Sheri Osborn, Emma Majors, Jeremy Davis, Anthony Rios
| Challenge: | Named Entity Recognition (NER) is particularly affected by noise, often termed the ASR-NLP gap. |
| Approach: | They propose a dataset to bridge the ASR-NLP gap in the biomedical domain by extracting adverse drug reactions and mentions of entities from the Brief Test of Adult Cognition by Telephone (BTACT) exam. |
| Outcome: | The proposed method can clean 2,000 clean and noisy recordings and eliminate errors using zero-shot and few-shot methods. |
Similar Papers
Why Aren’t We NER Yet? Artifacts of ASR Errors in Named Entity Recognition in Spontaneous Speech Transcripts (2023.acl-long)
Copied to clipboard
Piotr Szymański, Lukasz Augustyniak, Mikolaj Morzy, Adrian Szymczak, Krzysztof Surdyk, Piotr Żelasko
| Challenge: | despite advances in language models, the transcript of spontaneous human-human conversations remains an insurmountable challenge for most models. |
| Approach: | They examine the relationship between ASR and NER errors which limit NER models' ability to recover entity mentions from spontaneous speech transcripts. |
| Outcome: | The proposed model fails even if no word errors are introduced by the ASR . the proposed model's performance deteriorates when applied to the ASL outputs . |
Audio De-identification - a New Entity Recognition Task (N19-2)
Copied to clipboard
Ido Cohn, Itay Laish, Genady Beryozkin, Gang Li, Izhak Shafran, Idan Szpektor, Tzvika Hartman, Avinatan Hassidim, Yossi Matias
| Challenge: | Named Entity Recognition (NER) is an important step in de-identification (de-ID) of medical records, many of which are recorded conversations between a patient and a doctor. |
| Approach: | They propose to use Named Entity Recognition (NER) to detect audio spans with entity mentions in medical records and then use it to evaluate the results. |
| Outcome: | The proposed pipeline is based on a large labeled segment of the Switchboard and Fisher audio datasets and compares it with a benchmark. |
Where do LLMs currently stand on biomedical NER in both clean and noisy settings ? (2026.findings-eacl)
Copied to clipboard
| Challenge: | despite advances in medicine, many diseases remain without effective treatments . clinical meta-analysis is essential for drug discovery and clinical research . |
| Approach: | They investigate the performance of large language models (LLMs) on biomedical NER tasks . findings suggest LLMs exhibit a notable degree of robustness to noise . |
| Outcome: | The proposed models are closing the performance gap with BERT-based models and demonstrate particular strengths in low-data settings. |
CleanCoNLL: A Nearly Noise-Free Named Entity Recognition Dataset (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing models achieve F1-scores comparable to or exceed noise level in CoNLL-03 . current models have significant annotation errors, incompleteness, and inconsistencies in the data . |
| Approach: | They propose to add a layer of entity linking annotation to the CoNLL-03 corpus to correct 7.0% of all labels. |
| Outcome: | The proposed approach corrects 7.0% of all labels in the English CoNLL-03 dataset. |
Entity Decomposition with Filtering: A Zero-Shot Clinical Named Entity Recognition Framework (2025.naacl-long)
Copied to clipboard
| Challenge: | Recent studies have demonstrated that large language models (LLMs) can perform in named entity recognition tasks. |
| Approach: | They propose a framework for clinical named entity recognition that decomposes the entity recognition task into several retrievals of sub-types and then filters them. |
| Outcome: | The proposed framework improves on the clinical named entity recognition task. |
Chinese Spoken Named Entity Recognition in Real-world Scenarios: Dataset and Approaches (2024.findings-acl)
Copied to clipboard
| Challenge: | Current Chinese Spoken NER datasets are laboratory-controlled and are limited in topics. |
| Approach: | They propose to use Chinese Spoken NER datasets to extract entities from speech to help voice assistants better grasp the intent behind user's questions and instructions. |
| Outcome: | The proposed methods improve on self-training-asr and mapping then distilling, and even compared with GPT4.0, they achieve better results. |
Optimizing Entity Resolution in Voice Interfaces: An ASR-Aware Entity Reference Expansion Approach (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Automatic Speech Recognition (ASR) errors in voice-based dialog systems pose significant impediments to downstream tasks. |
| Approach: | They propose an automatic speech recognition (ASR) error-aware loss function to inject failed mentions and resolved entity names into the knowledge graph to enhance its awareness of unresolved mentions. |
| Outcome: | The proposed system enhances the knowledge graph's awareness of unresolved mentions by injecting pairs of failed mentions and resolved entities into the knowledge map. |
Evaluating Open-Source ASR Systems: Performance Across Diverse Audio Conditions and Error Correction Methods (2025.coling-main)
Copied to clipboard
| Challenge: | Automated speech recognition (ASR) systems are able to transcribe spontaneous human conversations with high accuracy. |
| Approach: | They evaluate the accuracy of open source automatic speech recognition systems across conversational speech datasets and explore the potential of ASR ensembling and post-ASR correction methods to improve transcription accuracy. |
| Outcome: | The proposed methods highlight the need for robust error correction techniques and address demographic biases to enhance ASR performance and inclusivity. |
Linguistically Informed Relation Extraction and Neural Architectures for Nested Named Entity Recognition in BioNLP-OST 2019 (D19-57)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) and Relation Extraction (RE) are essential tools in distilling knowledge from biomedical literature. |
| Approach: | They propose to use Named Entities to perform nested entities extraction, Entity Normalization and Relation Extraction to generalize the approach to different languages. |
| Outcome: | The proposed approach can be generalized to different languages and showed it’s effectiveness for English and Spanish text. |
The Medical Scribe: Corpus Development and Model Performance Analyses (2020.lrec-1)
Copied to clipboard
Izhak Shafran, Nan Du, Linh Tran, Amanda Perry, Lauren Keyes, Mark Knichel, Ashley Domin, Lei Huang, Yu-hui Chen, Gang Li, Mingqiu Wang, Laurent El Shafey, Hagen Soltau, Justin Stuart Paul
| Challenge: | Existing tools to assist in clinical note generation using audio of provider-patient encounters are lacking. |
| Approach: | They develop an annotation scheme to extract relevant clinical concepts from audio of provider-patient encounters and train a state-of-the-art tagging model. |
| Outcome: | The proposed model is more useful than the F-scores reflect and can be used in clinical notes. |