Papers with intelligibility
Towards Codec-LM Co-design for Neural Codec Language Models (2025.naacl-srw)
Copied to clipboard
| Challenge: | Neural codec language models (or codec LMs) are emerging as a powerful framework for text-to-speech (TTS) despite the close interdependence of codecs and LM, research on codec and lms has largely remained siloed. |
| Approach: | They propose a frame-wise codec encoder that improves both LM log-likelihood and TTS metrics . they also propose LM codebook level dropout to efficiently navigate a portion of codec-LM design space . |
| Outcome: | The proposed codec-LM co-design improves intelligibility, audio quality and speaker control compared to a siloed baseline. |
Automatic Song Translation for Tonal Languages (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing automatic song translation systems for tonal languages do not match the number of notes and beat the original rhythm of the song. |
| Approach: | They propose three criteria for effective AST: preserving meaning, singability and intelligibility. |
| Outcome: | The proposed system balances semantics and singability with human evaluations. |
VoiceStar: Robust Zero-Shot Autoregressive TTS with Duration Control and Extrapolation (2026.findings-acl)
Copied to clipboard
| Challenge: | Neural codec language models (NCLMs) lack fine-grained controllability and inability to extrapolate to sequence lengths much longer than those seen during training. |
| Approach: | They propose a novel autoregressive encoder-decoder neural codec language model that can be trained with a Continuation-Prompt Mixed training system. |
| Outcome: | The proposed model outperforms or is on par with current state-of-the-art models on short-form benchmarks such as LibriSpeech and Seed-TTS in terms of intelligibility and naturalness. |
Evaluating Text-to-Speech Synthesis from a Large Discrete Token-based Speech Language Model (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent advances in generative language modeling applied to discrete speech tokens presented a new avenue for text-to-speech (TTS) synthesis. |
| Approach: | They propose to use generative language modeling to generate text-to-speech (TTS) outputs by a discrete token-based model. |
| Outcome: | The proposed model is rated higher in naturalness and context appropriateness in listening tests compared to a conventional TTS. |
UniCoM: A Universal Code-Switching Speech Generator (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Code-switching (CS) is a common phenomenon in real-world conversations and poses significant challenges for multilingual speech technology. |
| Approach: | They propose a pipeline for generating high-quality, natural CS samples without altering sentence semantics. |
| Outcome: | The proposed pipeline generates high-quality, natural CS samples without altering sentence semantics without alteration of sentence semantic. |
Koel-TTS: Enhancing LLM based Speech Generation with Preference Alignment and Classifier Free Guidance (2025.emnlp-main)
Copied to clipboard
Shehzeen Samarah Hussain, Paarth Neekhara, Xuesong Yang, Edresson Casanova, Subhankar Ghosh, Roy Fejgin, Mikyas T. Desta, Rafael Valle, Jason Li
| Challenge: | Autoregressive speech token generation models suffer from hallucinations and undesired vocalizations that do not conform to conditioning inputs. |
| Approach: | They propose an encoder-decoder transformer model that improves contextual adherence of speech token generation LLMs through preference alignment and classifier-free guidance. |
| Outcome: | The proposed model outperforms previous LLM-based models on intelligibility, speaker similarity and naturalness. |