Versatile Framework for Song Generation with Prompt-based Control (2025.findings-emnlp)
Copied to clipboard
Yu Zhang, Wenxiang Guo, Changhao Pan, Zhiyuan Zhu, Ruiqi Li, Jingyu Lu, Rongjie Huang, Ruiyuan Zhang, Zhiqing Hong, Ziyue Jiang, Zhou Zhao
| Challenge: | Existing methods for song generation fail to generate vocals with prompt-based control and proper alignment. |
| Approach: | VersBand is a multi-task song generation framework for synthesizing high-quality songs with prompt-based control. |
| Outcome: | Experimental results show that VersBand performs better than baseline models across multiple song generation tasks. |
Similar Papers
Text-to-Song: Towards Controllable Music Generation Incorporating Vocal and Accompaniment (2024.acl-long)
Copied to clipboard
Zhiqing Hong, Rongjie Huang, Xize Cheng, Yongqi Wang, Ruiqi Li, Fuming You, Zhou Zhao, Zhimeng Zhang
| Challenge: | Existing studies focus on singing voice synthesis and music generation independently. |
| Approach: | They propose a novel task called Text-to-Song synthesis which incorporates both vocal and accompaniment generation. |
| Outcome: | The proposed method can synthesize songs with comparable quality and style consistency. |
Unsupervised Melody-to-Lyrics Generation (2023.acl-long)
Copied to clipboard
Yufei Tian, Anjali Narayan-Chen, Shereen Oraby, Alessandra Cervone, Gunnar Sigurdsson, Chenyang Tao, Wenbo Zhao, Tagyoung Chung, Jing Huang, Nanyun Peng
| Challenge: | Existing methods for automatic melody-to-lyric generation are limited due to the limited amount of melody-lyrical aligned data. |
| Approach: | They propose a method for automatic melody-to-lyric generation without training on any aligned melody-lyr data. |
| Outcome: | The proposed model generates high-quality lyrics that are singable, intelligible, and coherent than baseline models. |
SegTune: Structured and Fine-Grained Control for Song Generation (2026.acl-long)
Copied to clipboard
Yuejiao Wang, Zihao Ji, Pengfei Cai, Xu Li, Haorui Zheng, Zewen Song, Zhongliang Liu, Chen Zhang, Pengfei Wan
| Challenge: | Recent advances in neural song generation have enabled high-quality synthesis from lyrics and global textual prompts. |
| Approach: | They propose a framework that allows users to specify local musical descriptions aligned to song segments. |
| Outcome: | The proposed framework outperforms baselines in musicality and controllability. |
Prompt-Singer: Controllable Singing-Voice-Synthesis with Natural Language Prompt (2024.naacl-long)
Copied to clipboard
Yongqi Wang, Ruofan Hu, Rongjie Huang, Zhiqing Hong, Ruiqi Li, Wenrui Liu, Fuming You, Tao Jin, Zhou Zhao
| Challenge: | Recent singing-voice-synthesis methods lack ability to control style attributes of synthesized singing. |
| Approach: | They propose a singing-voice-synthesis method that enables attribute controlling on singer gender, vocal range and volume with natural language. |
| Outcome: | The proposed method achieves favorable control ability and audio quality. |
Generative Music Models’ Alignment with Professional and Amateur Users’ Expectations (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent years have witnessed rapid advances in text-to-music generation using large language models. |
| Approach: | They propose a task to align AI-generated music with human expressions . they use a dataset of over 1.5 million songs to analyze their content . |
| Outcome: | The proposed framework outperforms baseline models and facilitates end-to-end generation of songs audio. |
SongComposer: A Large Language Model for Lyric and Melody Generation in Song Composition (2025.acl-long)
Copied to clipboard
Shuangrui Ding, Zihan Liu, Xiaoyi Dong, Pan Zhang, Rui Qian, Junhao Huang, Conghui He, Dahua Lin, Jiaqi Wang
| Challenge: | Creating lyrics and melodies in symbolic format requires expert knowledge of melody and an advanced understanding of lyrics. |
| Approach: | They introduce SongComposer, a music-specialized large language model that can create symbolic lyrics and melodies following instructions. |
| Outcome: | The proposed model outperforms existing models in symbolic song composition tasks. |
TCSinger 2: Customizable Multilingual Zero-shot Singing Voice Synthesis (2025.findings-acl)
Copied to clipboard
Yu Zhang, Wenxiang Guo, Changhao Pan, Dongyu Yao, Zhiyuan Zhu, Ziyue Jiang, Yuhan Wang, Tao Jin, Zhou Zhao
| Challenge: | Existing zero-shot singing voice synthesis models depend on phoneme and note boundary annotations, limiting their robustness and producing poor transitions between phonemes and notes. |
| Approach: | They propose a multi-task multilingual zero-shot SVS model with style transfer and style control based on various prompts. |
| Outcome: | Experimental results show that TCSinger 2 outperforms baseline models in subjective and objective metrics across multiple related tasks. |
VocalRep: Structure-Aware Vocal Representations for Multimodal Generation (2026.findings-acl)
Copied to clipboard
Da Shen, Zhenqiang Weng, Tianyu Liu, Gongyu Chen, Runhua Shi, Jiahui Chen, Chaofan Ding, Wei-Qiang Zhang, Zihao Chen
| Challenge: | Existing approaches to vocal separation are optimized for signal-level reconstruction, but they overlook structural disentanglement required for downstream generation tasks. |
| Approach: | They propose a structure-aware learning framework to disentangle vocals, harmonies, and accompaniment . they combine global vocal identity conditioning with ranking-based objectives . |
| Outcome: | The proposed framework disentangles lead vocals, harmonies, and accompaniment while enforcing role consistency across long-form audio. |
UniLG: A Unified Structure-aware Framework for Lyrics Generation (2023.acl-long)
Copied to clipboard
| Challenge: | Existing works ignore musical attributes hidden behind lyrics and structure of lyrics . existing works ignore structure of generated lyrics and do not consider structure of songs . |
| Approach: | They propose a framework for conditional lyrics generation that considers structure and relationship between lyrics and music. |
| Outcome: | The proposed framework improves the structure modeling and unifies different conditions for different types of lyrics generation. |
SongRewriter: A Chinese Song Rewriting System with Controllable Content and Rhyme Scheme (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods of generating singable lyrics are based on a given melody, but there are two main challenges: generating the lyrics without knowing the melody and composing compatible melodies. |
| Approach: | They propose a Chinese lyric generation and editing system which rewrites lyrics of an existing song such that they are compatible with the rhythm of the existing melody. |
| Outcome: | The proposed system is based on a randomized multi-level masking strategy and can generate new lyrics or edit fragments without prior knowledge of melody composition. |