Papers by Puyuan Peng
VoiceCraft: Zero-Shot Speech Editing and Text-to-Speech in the Wild (2024.acl-long)
Copied to clipboard
| Challenge: | VoiceCraft is a token-infilling neural codec language model for speech editing and zero-shot text-to-speech evaluation. |
| Approach: | They introduce a token infilling neural codec language model that performs on speech editing and zero-shot text-to-speech tasks. |
| Outcome: | The proposed model outperforms previous models on speech editing and zero-shot text-to-speech tasks. |
VoiceStar: Robust Zero-Shot Autoregressive TTS with Duration Control and Extrapolation (2026.findings-acl)
Copied to clipboard
| Challenge: | Neural codec language models (NCLMs) lack fine-grained controllability and inability to extrapolate to sequence lengths much longer than those seen during training. |
| Approach: | They propose a novel autoregressive encoder-decoder neural codec language model that can be trained with a Continuation-Prompt Mixed training system. |
| Outcome: | The proposed model outperforms or is on par with current state-of-the-art models on short-form benchmarks such as LibriSpeech and Seed-TTS in terms of intelligibility and naturalness. |
VoiceCraft-X: Unifying Multilingual, Voice-Cloning Speech Synthesis and Speech Editing (2025.emnlp-main)
Copied to clipboard
Zhisheng Zheng, Puyuan Peng, Anuj Diwan, Cong Phuoc Huynh, Xiaohang Sun, Zhu Liu, Vimal Bhat, David Harwath
| Challenge: | Autoregressive language model for multilingual speech editing and zero-shot text-to-speech synthesis is available in 11 languages. |
| Approach: | They introduce an autoregressive neural codec language model which unifies multilingual speech editing and zero-shot text-to-speech synthesis across 11 languages. |
| Outcome: | The model generates high-quality, natural-sounding speech, even with limited per-language data . it shows robust performance in diverse linguistic settings, even in limited per language data compared to other models . |