ControlAudio: Tackling Text-Guided, Timing-Indicated and Intelligible Audio Generation via Progressive Diffusion Modeling (2026.acl-long)
Copied to clipboard
| Challenge: | Recent efforts on text-to-audio generation are exploring fine-grained controllability . however, their performance at scale is limited due to data scarcity . |
| Approach: | They propose a multi-task learning problem for high-controllability text-to-audio generation . they propose scalable diffusion transformers that augment condition information in sequence . |
| Outcome: | The proposed method outperforms existing methods on objective and subjective evaluations. |
Similar Papers
FlashAudio: Rectified Flow for Fast and High-Fidelity Text-to-Audio Generation (2025.acl-long)
Copied to clipboard
| Challenge: | Recent advances in latent diffusion models (LDMs) have markedly enhanced text-to-audio generation, yet their iterative sampling processes impose substantial computational demands, limiting practical deployment. |
| Approach: | They propose to learn straight flow for fast simulation by using flashAudio with rectified flows and immiscible flow to minimize the total distance of data-noise pairs in a batch vias assignment. |
| Outcome: | The proposed method can learn straight flow for fast simulations and reduce noise distribution. |
UniSonate: A Unified Model for Speech, Music, and Sound Effect Generation with Text Instructions (2026.acl-long)
Copied to clipboard
Chunyu Qiang, Xiaopeng Wang, Kang Yin, Yuzhe Liang, Yuxin Guo, Teng Ma, Ziyu Zhang, Tianrui Wang, Cheng Gong, Yushen Chen, Ruibo Fu, Longbiao Wang, Jianwu Dang
| Challenge: | Generative audio modeling has been fragmented into specialized tasks such as text-to-speech (TTS), text- to-music (TTM), and text-ta (TTA) specialized models require reference audio for timbre cloning and strict phoneme alignment, whereas TTA models generate unstructured textures from open-ended captions. |
| Approach: | They propose a unified flow-matching framework capable of synthesizing speech, music, sound effects . they propose 'token injection mechanism' that projects unstructured environmental sounds into structured temporal latent space . |
| Outcome: | The proposed framework achieves state-of-the-art performance in instruction-based TTS and TTM while maintaining competitive fidelity in TTA. |
Mustango: Toward Controllable Text-to-Music Generation (2024.naacl-long)
Copied to clipboard
| Challenge: | Mustango is a text-to-music system that allows music-domain-knowledge-informed text-based music generation. |
| Approach: | They propose a music-domain-knowledge-inspired text-to-music system based on diffusion that generates music with captions that include specific instructions related to chords, beats, key and tempo. |
| Outcome: | The proposed system outperforms existing models in music generation tasks. |
SegTune: Structured and Fine-Grained Control for Song Generation (2026.acl-long)
Copied to clipboard
Yuejiao Wang, Zihao Ji, Pengfei Cai, Xu Li, Haorui Zheng, Zewen Song, Zhongliang Liu, Chen Zhang, Pengfei Wan
| Challenge: | Recent advances in neural song generation have enabled high-quality synthesis from lyrics and global textual prompts. |
| Approach: | They propose a framework that allows users to specify local musical descriptions aligned to song segments. |
| Outcome: | The proposed framework outperforms baselines in musicality and controllability. |
MeanAudio: Fast and Faithful Text-to-Audio Generation with Mean Flows (2026.acl-long)
Copied to clipboard
| Challenge: | Recent advances in Text-to-Audio Generation (TTA) systems suffer from slow inference speed, authors report . authors demonstrate that MeanAudia achieves state-of-the-art performance in single-step audio generation . |
| Approach: | They propose a text-to-audio generator capable of rendering realistic sound with only one function evaluation. |
| Outcome: | The proposed system achieves state-of-the-art performance in single-step audio generation. |
DiffLM: Controllable Synthetic Data Generation via Diffusion Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have significantly enhanced their knowledge and generative capabilities, leading to a surge of interest in leveraging LLMs for high-quality data synthesis. |
| Approach: | They propose a controllable data synthesis framework based on variational autoencoder which leverages diffusion models to reserve more information of original distribution and format structure in the learned latent distribution. |
| Outcome: | The proposed framework generates high-quality data with performance exceeding that of real data by 2%–7% on seven real-world datasets. |
Towards Controllable Speech Synthesis in the Era of Large Language Models: A Systematic Survey (2025.emnlp-main)
Copied to clipboard
| Challenge: | Text-to-speech (TTS) has advanced from generating natural-sounding speech to enabling fine-grained control over speech attributes. |
| Approach: | They provide a review of controllable TTS methods from traditional control techniques to emerging approaches using natural language prompts. |
| Outcome: | The proposed methods are based on models, strategies, and features, and summarize challenges, datasets, and evaluations. |
T2A-Feedback: Improving Basic Capabilities of Text-to-Audio Generation via Fine-grained AI Feedback (2025.acl-long)
Copied to clipboard
Zehan Wang, Ke Lei, Chen Zhu, Jiawei Huang, Sashuai Zhou, Luping Liu, Xize Cheng, Shengpeng Ji, Zhenhui Ye, Tao Jin, Zhou Zhao
| Challenge: | Text-to-audio (T2A) models still struggle to satisfy human preferences for prompt-following and acoustic quality when generating complex multi-event audio. |
| Approach: | They propose to use AI feedback learning to enhance basic capabilities of text-to-audio models . they use a large audio preference dataset to evaluate the model's capabilities . |
| Outcome: | The proposed model improves in simple and complex scenarios with AI feedback learning. |
RSA-Control: A Pragmatics-Grounded Lightweight Controllable Text Generation Framework (2024.emnlp-main)
Copied to clipboard
| Challenge: | RSA-Control is a training-free controllable text generation framework . existing studies rely on fine-tuning pre-trained language models . external components could hurt coherence and accuracy of the model . |
| Approach: | They propose a training-free controllable text generation framework grounded in pragmatics that directs the generation process by recursively reasoning between imaginary speakers and listeners. |
| Outcome: | The proposed framework achieves strong attribute control while maintaining fluency and content consistency. |
Moûsai: Efficient Text-to-Music Diffusion Models (2024.acl-long)
Copied to clipboard
| Challenge: | Recent years have seen the rapid development of large generative models for text; however, little research has explored the connection between text and another “language” of communication – music. |
| Approach: | They develop a text-to-music generation model that can generate multiple minutes of high-quality stereo music at 48kHz from textual descriptions. |
| Outcome: | The proposed model can generate multiple minutes of high-quality stereo music at 48kHz from textual descriptions. |