Papers with GSLM
Generative Spoken Language Model based on continuous word-sized audio tokens (2023.emnlp-main)
Copied to clipboard
| Challenge: | Text-based language models outperform character-based models, but speech inputs are 20ms or 40ms-long discrete units. |
| Approach: | They propose a generative language model based on word-size continuous audio tokens . they replace lookup table for lexical types with a Lexical Embedding function . |
| Outcome: | The proposed model is five times more memory efficient than discrete unit GSLMs and is phonetically and semantically interpretable. |
Text-Free Prosody-Aware Generative Spoken Language Modeling (2022.acl-long)
Copied to clipboard
Eugene Kharitonov, Ann Lee, Adam Polyak, Yossi Adi, Jade Copet, Kushal Lakhotia, Tu Anh Nguyen, Morgane Riviere, Abdelrahman Mohamed, Emmanuel Dupoux, Wei-Ning Hsu
| Challenge: | Experimental results show that generative spoken language models (LMs) are natural unsupervised multitask learners. |
| Approach: | They propose a prosody-aware generative spoken language model that uses discovered units to generate natural, meaningful, and coherent speech. |
| Outcome: | The proposed model can generate natural, meaningful, and coherent speech given a spoken prompt. |