PyramidCodec: Hierarchical Codec for Long-form Music Generation in Audio Domain (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to generate long music are inefficient and lack of structured representation. |
| Approach: | They propose a hierarchical discrete representation of audio for long audio-domain music generation using residual vector quantization on different levels of features. |
| Outcome: | The proposed method achieves competitive performance in terms of reconstruction quality and token per second (TPS) the proposed method facilitates training a language model that can generate well-structured long-form music for up to 3 minutes. |
Similar Papers
Hierarchical Representation Alignment Learning of Diffusion Transformers for Neural Audio Codec (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in diffusion and conditional flow matching models for low-resolution domains are underexplored. |
| Approach: | They propose a CFM-based model that iteratively generates raw waveform in low-bitrate conditions . they propose DVQ, a factorized quantization method that uses a single quantizer . |
| Outcome: | The proposed model outperforms state-of-the-art neural audio codecs in audio quality and semantic intelligibility under low-bitrate conditions. |
Language-Codec: Bridging Discrete Codec Representations and Speech Language Models (2025.acl-long)
Copied to clipboard
Shengpeng Ji, Minghui Fang, Jialong Zuo, Ziyue Jiang, Dingdong Wang, Hanting Wang, Hai Huang, Zhou Zhao
| Challenge: | Existing gaps between discrete acoustic codecs and downstream speech language models . initial channel of codebooks contains excessive information, making it difficult to generate tokens from weakly supervised signals such as text. |
| Approach: | They propose a discrete acoustic codec for generating acustic tokens from weakly supervised signals. |
| Outcome: | The proposed language-codec outperforms competing audio compression algorithms and validates on downstream speech language models. |
Anchored Cyclic Generation: A Novel Paradigm for Long-Sequence Symbolic Music Generation (2026.findings-acl)
Copied to clipboard
| Challenge: | Autoregressive sequence modeling has been successful in many domains, but maintaining long-term coherence and structural integrity remains a challenge. |
| Approach: | They propose an ACG paradigm that relies on anchor features from previously generated musical content to guide subsequent generation during the autoregressive process. |
| Outcome: | The proposed framework outperforms existing methods in symbolic music generation tasks. |
Muse: Towards Reproducible Long-Form Song Generation with Fine-Grained Style Control (2026.findings-acl)
Copied to clipboard
Changhao Jiang, Jiahao Chen, Zhenghao Xiang, Zhixiong Yang, Hanchen Wang, Jiabao Zhuang, Xinmeng Che, Jiajun Sun, Hui Li, Yifei Cao, Shihan Dou, Ming Zhang, Junjie Ye, Tao Ji, Tao Gui, Qi Zhang, Xuanjing Huang
| Challenge: | Recent commercial systems such as Suno demonstrate strong capabilities in long-form song generation, but academic research remains non-reproducible due to the lack of publicly available training data. |
| Approach: | They propose a system for long-form song generation with fine-grained style conditioning that includes a licensed synthetic dataset and a song generation model, Muse. |
| Outcome: | The proposed system achieves competitive performance on phoneme error rate, text–music style similarity, and audio aesthetic quality while enabling controllable segment-level generation across different musical structures. |
RepCodec: A Speech Representation Codec for Speech Tokenization (2024.acl-long)
Copied to clipboard
| Challenge: | Recent advances in large language models have led to discrete speech tokenization, but this discretization can be costly and impedes performance. |
| Approach: | They propose a new speech representation codec for semantic speech tokenization that reconstructs speech representations from speech encoders like HuBERT or data2vec. |
| Outcome: | The proposed method outperforms the widely used k-means clustering approach in speech understanding and generation. |
Generative Pre-trained Speech Language Model with Efficient Hierarchical Transformer (2024.acl-long)
Copied to clipboard
| Challenge: | Experimental results indicate that GPST significantly outperforms the existing speech language models in terms of word error rate, speech quality, and speaker similarity. |
| Approach: | They propose a hierarchical transformer that quantizes audio waveforms into two distinct types of discrete speech representations and integrates them within a transformer architecture. |
| Outcome: | The proposed model outperforms existing speech language models in word error rate, speech quality, and speaker similarity. |
Hierarchical Bracketing Encodings Work for Dependency Graphs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Sequence labeling (SL) is a simple yet effective paradigm for a wide range of natural language problems. |
| Approach: | They propose a new bracketing approach for dependency graph parsing that encodes graphs as sequences and n tagging actions. |
| Outcome: | The proposed approach significantly reduces label space while preserving structural information. |
Codec-SUPERB: An In-Depth Analysis of Sound Codec Models (2024.findings-acl)
Copied to clipboard
Haibin Wu, Ho-Lam Chung, Yi-Cheng Lin, Yuan-Kuei Wu, Xuanjun Chen, Yu-Chi Pai, Hsiu-Hsuan Wang, Kai-Wei Chang, Alexander Liu, Hung-yi Lee
| Challenge: | Researchers have developed a sound codec that can be used as tokenizers for preserving audio data and minimizing data transmission latency. |
| Approach: | They propose to use codec-SUPERB to assess codec models across representative sound applications and signal-level metrics rooted in sound domain knowledge. |
| Outcome: | The proposed codec-SUPERB model is evaluated on selected experimental settings. |
HiGen: Hierarchy-Aware Sequence Generation for Hierarchical Text Classification (2024.eacl-long)
Copied to clipboard
| Challenge: | Hierarchical text classification is a complex subtask under multi-label text classification . the relevance of document sections can vary based on the hierarchy level, necessitating a dynamic document representation. |
| Approach: | They propose a text-generation-based framework that uses language models to encode dynamic text representations. |
| Outcome: | The proposed framework surpasses existing methods while handling data and mitigating class imbalance. |
UnitCoder: Scalable Code Synthesis from Pre-training Corpora (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) excel at code understanding and generation, yet code generation remains a challenge. |
| Approach: | They propose a model that supervises pre-training data quality through automatically generated unit tests while ensuring correctness via an iterative fix and refine flow. |
| Outcome: | The proposed model improves performance on a large dataset with high quality pre-training data. |