emg2speech: synthesizing speech from electromyography using self-supervised speech models (2026.acl-long)
Copied to clipboard
| Challenge: | a neuromuscular speech interface translates electromyographic (EMG) signals recorded from orofacial muscles during speech articulation directly into audio. |
| Approach: | They propose a neuromuscular speech interface that translates electromyographic (EMG) signals recorded from orofacial muscles during speech articulation directly into audio. |
| Outcome: | The proposed system synthesizes speech without explicit modeling or vocoder training. |
Similar Papers
Non-invasive electromyographic speech neuroprosthesis: a geometric perspective (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for mapping EMG to time-aligned audio limits applicability to patients who can no longer speak. |
| Approach: | They propose a neuromuscular speech interface that translates silently voiced articulations directly into text. |
| Outcome: | The proposed interface can translate silently voiced articulations directly into text without audio transfer . the proposed system could restore communication for patients with speech loss . |
Can LLMs Understand Unvoiced Speech? Exploring EMG-to-Text Conversion with LLMs (2025.acl-short)
Copied to clipboard
| Challenge: | Unvoiced electromyography (EMG) is an effective communication tool for individuals unable to produce vocal speech. |
| Approach: | They propose an EMG adaptor module that maps EMG features to an LLM's input space and achieves an average word error rate of 0.49 on a closed-vocabulary unvoiced EMG-to-text task. |
| Outcome: | The proposed module achieves an average word error rate of 0.49 on a closed-vocabulary unvoiced EMG-to-text task. |
An Improved Model for Voicing Silent Speech (2021.acl-short)
Copied to clipboard
| Challenge: | Existing models for voicing silent speech use hand-designed features instead of EMG signals. |
| Approach: | They propose to use facial electromyography signals as input instead of hand-designed features to give the model greater flexibility to learn its own features. |
| Outcome: | The proposed model improves state-of-the-art on an open vocabulary intelligibility evaluation by 25.8%. |
Digital Voicing of Silent Speech (2020.emnlp-main)
Copied to clipboard
| Challenge: | Using electromyography, we can convert silently mouthed words into audible speech . prior work focused on training speech synthesis models from vocalized data . |
| Approach: | They propose a method of training on silent EMG by transferring audio targets from vocalized to silent signals and propose voicing task using muscle sensor measurements. |
| Outcome: | The proposed method greatly improves intelligibility of audio generated from silent EMG compared to baseline that only trains with vocalized data. |
[b] = [d] - [t] + [p]: Self-supervised Speech Models Discover Phonological Vector Arithmetic (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on how self-supervised speech models encode rich phonetic information have not explored how they are structured. |
| Approach: | They conduct a comprehensive analysis of the underlying structure of S3M representations with particular attention to phonological vectors. |
| Outcome: | The proposed model encodes phonologically interpretable and compositional vectors, demonstrating phonology vector arithmetic. |
From perception to production: how acoustic invariance facilitates articulatory learning in a self-supervised vocal imitation model (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing models that map variable acoustic inputs into appropriate articulatory movements without explicit instruction are inadequate for infants. |
| Approach: | They propose a model that maps acoustic inputs into articulatory movements without explicit instruction for infants. |
| Outcome: | The proposed model outperforms MFCC features in both single- and multi-speaker settings and provides optimal representations for articulatory learning. |
Emergent morpho-phonological representations in self-supervised speech models (2025.emnlp-main)
Copied to clipboard
| Challenge: | a recent study shows that self-supervised speech models do not represent phonological and morphological phenomena in frequent English noun and verb inflections. |
| Approach: | They study how S3Ms represent phonological and morphological phenomena in English . they propose alternative representational strategies that may support human spoken word recognition . |
| Outcome: | a new study shows that S3M models can represent phonological and morphological phenomena in English . the models can be trained to recognize spoken words in naturalistic, noisy environments . |
EMGLLM: Data-to-Text Alignment for Electromyogram Diagnosis Generation with Medical Numerical Data Encoding (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing Large Language Models struggle to interpret EMG tables . EMGLLM is a data-to-text model for medical examination tables based on electrical signals . |
| Approach: | They propose a data-to-text model that aligns EMG data into word embeddings that reflect health degree. |
| Outcome: | The proposed model outperforms baseline models in understanding EMG tables and generating high-quality diagnoses. |
Emotags: Computer-Assisted Verbal Labelling of Expressive Audiovisual Utterances for Expressive Multimodal TTS (2024.lrec-main)
Copied to clipboard
| Challenge: | We show that ascribing verbal descriptions to expressive audiovisual utterances is efficient and efficient. |
| Approach: | They propose a web app for ascribing verbal descriptions to expressive audiovisual utterances. |
| Outcome: | The proposed system can be deployed at a large scale to efficiently collect relevant verbal descriptions. |
Self-supervised Representation Learning for Speech Processing (2022.naacl-tutorials)
Copied to clipboard
Hung-yi Lee, Abdelrahman Mohamed, Shinji Watanabe, Tara Sainath, Karen Livescu, Shang-Wen Li, Shu-wen Yang, Katrin Kirchhoff
| Challenge: | Self-supervised representation learning (SSL) uses proxy supervised learning tasks to obtain training data from unlabeled corpora. |
| Approach: | They propose to survey the latest SSL techniques, tools, datasets, and performance achievement in speech processing to scale up current machine learning technologies. |
| Outcome: | The proposed tutorial is highly relevant to the special theme of ACL about language diversity. |