RMSSinger: Realistic-Music-Score based Singing Voice Synthesis

Challenge:	Existing methods for singing voice synthesis are limited to fine-grained music scores . manual adjustment destroys regularity of note durations, making fine-grain music scores "crushed"
Approach:	They propose a method to synthesize singing voices given realistic music scores . they use real-music-score-based Singing Voice Synthesis to generate high-quality voices .
Outcome:	The proposed method eliminates manual annotation and simplifies phoneme-level mel-note alignment.

Similar Papers

Overview

Challenge

Approach

Outcome

STARS: A Unified Framework for Singing Transcription, Alignment, and Refined Style Annotation (2025.findings-acl)

Copied to clipboard

Wenxiang Guo, Yu Zhang, Changhao Pan, Zhiyuan Zhu, Ruiqi Li, ZheTao Chen, Wenhao Xu, Fei Wu, Zhou Zhao

Speech & Audio in NLP

Challenge:	Existing automated singing annotation (ASA) methods tackle isolated aspects of the annotation pipeline.
Approach:	They propose a framework that addresses transcription, alignment, and refined style annotations.
Outcome:	The proposed framework delivers comprehensive multi-level annotations encompassing: (1) precise phoneme-audio alignment, (2) robust note transcription and temporal localization, (3) expressive vocal technique identification, and (4) global stylistic characterization including emotion and pace.

Robust Singing Voice Transcription Serves Synthesis (2024.acl-long)

Copied to clipboard

Ruiqi Li, Yu Zhang, Yongqi Wang, Zhiqing Hong, Rongjie Huang, Zhou Zhao

Robustness in NLP

Challenge:	Current AST methods struggle with accuracy and robustness when used for practical annotation.
Approach:	They propose a model that converts singing recordings into note sequences for automatic annotation of singing datasets.
Outcome:	The proposed model outperforms baseline models on enlarged, automatically annotated datasets.

AlignSTS: Speech-to-Singing Conversion via Cross-Modal Alignment (2023.findings-acl)

Copied to clipboard

Ruiqi Li, Rongjie Huang, Lichao Zhang, Jinglin Liu, Zhou Zhao

Multimodality

Challenge:	Existing approaches to speech-to-singing voice conversion are difficult to learn in text-free situations.
Approach:	They propose an STS model which views speech variance as different modalities . it uses a novel rhythm adaptor to predict the target rhythm representation . they also use the predicted rhythm representation to re-align the content .
Outcome:	The proposed model achieves superior performance in terms of objective and subjective metrics.

Self-Supervised Singing Voice Pre-Training towards Speech-to-Singing Conversion (2024.findings-acl)

Copied to clipboard

Ruiqi Li, Rongjie Huang, Yongqi Wang, Zhiqing Hong, Zhou Zhao

Speech & Audio in NLP

Challenge:	Existing studies on speech-to-singing voice conversion (STS) are limited by the scarcity of paired speech-song data and the suboptimal quality of outputs.
Approach:	They propose a self-supervised singing voice pre-training model that transforms a speech-to-singing voice into a paired singing voice.
Outcome:	The proposed model improves both STS and singing voice synthesis tasks by combining spoken language and a self-supervised singing voice pre-training model.

Improving Chinese Pop Song and Hokkien Gezi Opera Singing Voice Synthesis by Enhancing Local Modeling (2023.emnlp-main)

Copied to clipboard

Peng Bai, Yue Zhou, Meizhen Zheng, Wujin Sun, Xiaodong Shi

Speech & Audio in NLP

Challenge:	Singing Voice Synthesis (SVS) synthesizes pleasing vocals based on music scores and lyrics . current acoustic models ignore the significance of local modeling within the sequence and the hard-to-synthesize parts in the predicted mel-spectrogram .
Approach:	They propose a method to enhance local modeling in the acoustic model by focusing on phoneme tokens located before and after the phoneme.
Outcome:	The proposed method improves local modeling in the acoustic model by focusing on the hard-to-synthesize parts of the predicted mel-spectrogram.

TCSinger 2: Customizable Multilingual Zero-shot Singing Voice Synthesis (2025.findings-acl)

Copied to clipboard

Yu Zhang, Wenxiang Guo, Changhao Pan, Dongyu Yao, Zhiyuan Zhu, Ziyue Jiang, Yuhan Wang, Tao Jin, Zhou Zhao

Responsible & Trustworthy NLP

Challenge:	Existing zero-shot singing voice synthesis models depend on phoneme and note boundary annotations, limiting their robustness and producing poor transitions between phonemes and notes.
Approach:	They propose a multi-task multilingual zero-shot SVS model with style transfer and style control based on various prompts.
Outcome:	Experimental results show that TCSinger 2 outperforms baseline models in subjective and objective metrics across multiple related tasks.

Prompt-Singer: Controllable Singing-Voice-Synthesis with Natural Language Prompt (2024.naacl-long)

Copied to clipboard

Yongqi Wang, Ruofan Hu, Rongjie Huang, Zhiqing Hong, Ruiqi Li, Wenrui Liu, Fuming You, Tao Jin, Zhou Zhao

Language Models

Challenge:	Recent singing-voice-synthesis methods lack ability to control style attributes of synthesized singing.
Approach:	They propose a singing-voice-synthesis method that enables attribute controlling on singer gender, vocal range and volume with natural language.
Outcome:	The proposed method achieves favorable control ability and audio quality.

Learning the Beauty in Songs: Neural Singing Voice Beautifier (2022.acl-long)

Copied to clipboard

Jinglin Liu, Chengxi Li, Yi Ren, Zhiying Zhu, Zhou Zhao

Speech & Audio in NLP

Challenge:	Existing techniques for pitch correction are limited to intonation but ignore the overall aesthetic quality.
Approach:	They propose a novel time-warping approach for pitch correction to synchronize the amateur recording with the template pitch curve.
Outcome:	The proposed model improves intonation and vocal tone while keeping content and vocal timbre.

TCSinger: Zero-Shot Singing Voice Synthesis with Style Transfer and Multi-Level Style Control (2024.emnlp-main)

Copied to clipboard

Yu Zhang, Ziyue Jiang, Ruiqi Li, Changhao Pan, Jinzheng He, Rongjie Huang, Chuxin Wang, Zhou Zhao

Low-Resource NLP

Challenge:	Existing models fail to generate singing voices rich in stylistic nuances for unseen singers due to multifaceted nature of singing styles.
Approach:	They propose a zero-shot SVS model for style transfer across cross-lingual speech and singing styles and multi-level style control.
Outcome:	Experimental results show that TCSinger outperforms baseline models in synthesis quality, singer similarity, and style controllability.

ResoDiff-44k: High-Fidelity Cross-Lingual Speech and Singing Synthesis via Discrete Diffusion (2026.acl-industry)

Copied to clipboard

Gyanendra Das, Sai Satyam Jena

Speech & Audio in NLP

Challenge:	generative speech models have a fidelity ceiling that is capped at lower sampling rates . current models rely on intermediate mel-spectrograms, which discard phase and high-frequency information . a new framework that synthesizes industrial-grade 44.1kHz audio is proposed .
Approach:	They propose a production-grade generative foundation model for 44.1kHz audio synthesis . they pre-train ResoDiff-44k on a massive 150K -hour multilingual dataset .
Outcome:	The proposed model achieves 4.6 mean opinion score in 44.1kHz singing synthesis compared to baselines . it also reduces character error rate on regional mixed-language and singing prompts compared with baselines.