Papers by Zhiqing Hong
AudioVSR: Enhancing Video Speech Recognition with Audio Data (2024.emnlp-main)
Copied to clipboard
Xiaoda Yang, Xize Cheng, Jiaqi Duan, Hongshun Qiu, Minjie Hong, Minghui Fang, Shengpeng Ji, Jialong Zuo, Zhiqing Hong, Zhimeng Zhang, Tao Jin
| Challenge: | Recent work has shown poor performance with non-Indo-European languages . previous work primarily utilizes video information to build VSR models . |
| Approach: | They propose a generative model for data inflation that integrates synthetic data with authentic visual data to enhance the VSR model. |
| Outcome: | The proposed model improves on the audio-visual alignment problem in audio-video tasks. |
Speech-to-Speech Translation with Discrete-Unit-Based Style Transfer (2024.acl-srw)
Copied to clipboard
| Challenge: | Existing methods to translate spoken utterances from one language to another are unable to preserve speaker timbre of source speech. |
| Approach: | They propose a pipeline with style-transfer capability on the basis of self-supervised speech representations and codec units. |
| Outcome: | The proposed model achieves zero-shot cross-lingual style transfer on previously unseen source languages. |
Variational Language Concepts for Interpreting Foundation Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Foundation Language Models (FLMs) have achieved remarkable success in natural language processing. |
| Approach: | They propose a variational Bayesian framework to provide word-level interpretations for FLMs . they propose valc to find optimal language concepts to interpret FLM predictions . |
| Outcome: | Empirical results show that the proposed framework can provide conceptual interpretations for foundation language models. |
Versatile Framework for Song Generation with Prompt-based Control (2025.findings-emnlp)
Copied to clipboard
Yu Zhang, Wenxiang Guo, Changhao Pan, Zhiyuan Zhu, Ruiqi Li, Jingyu Lu, Rongjie Huang, Ruiyuan Zhang, Zhiqing Hong, Ziyue Jiang, Zhou Zhao
| Challenge: | Existing methods for song generation fail to generate vocals with prompt-based control and proper alignment. |
| Approach: | VersBand is a multi-task song generation framework for synthesizing high-quality songs with prompt-based control. |
| Outcome: | Experimental results show that VersBand performs better than baseline models across multiple song generation tasks. |
Prompt-Singer: Controllable Singing-Voice-Synthesis with Natural Language Prompt (2024.naacl-long)
Copied to clipboard
Yongqi Wang, Ruofan Hu, Rongjie Huang, Zhiqing Hong, Ruiqi Li, Wenrui Liu, Fuming You, Tao Jin, Zhou Zhao
| Challenge: | Recent singing-voice-synthesis methods lack ability to control style attributes of synthesized singing. |
| Approach: | They propose a singing-voice-synthesis method that enables attribute controlling on singer gender, vocal range and volume with natural language. |
| Outcome: | The proposed method achieves favorable control ability and audio quality. |
Self-Supervised Singing Voice Pre-Training towards Speech-to-Singing Conversion (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on speech-to-singing voice conversion (STS) are limited by the scarcity of paired speech-song data and the suboptimal quality of outputs. |
| Approach: | They propose a self-supervised singing voice pre-training model that transforms a speech-to-singing voice into a paired singing voice. |
| Outcome: | The proposed model improves both STS and singing voice synthesis tasks by combining spoken language and a self-supervised singing voice pre-training model. |
Robust Singing Voice Transcription Serves Synthesis (2024.acl-long)
Copied to clipboard
| Challenge: | Current AST methods struggle with accuracy and robustness when used for practical annotation. |
| Approach: | They propose a model that converts singing recordings into note sequences for automatic annotation of singing datasets. |
| Outcome: | The proposed model outperforms baseline models on enlarged, automatically annotated datasets. |
Text-to-Song: Towards Controllable Music Generation Incorporating Vocal and Accompaniment (2024.acl-long)
Copied to clipboard
Zhiqing Hong, Rongjie Huang, Xize Cheng, Yongqi Wang, Ruiqi Li, Fuming You, Zhou Zhao, Zhimeng Zhang
| Challenge: | Existing studies focus on singing voice synthesis and music generation independently. |
| Approach: | They propose a novel task called Text-to-Song synthesis which incorporates both vocal and accompaniment generation. |
| Outcome: | The proposed method can synthesize songs with comparable quality and style consistency. |
CoAlign: Uncertainty Calibration of LLM for Geospatial Repartition (2025.acl-industry)
Copied to clipboard
| Challenge: | Existing methods to optimize geospatial repartition rely on manual adjustments by experts or algorithmic optimization using limited offline operational metrics. |
| Approach: | They propose a framework that calibrates LLM uncertainty to enable robust geospatial repartition by integrating historical data with LLM-generated candidates. |
| Outcome: | The proposed framework calibrates LLM uncertainty to enable robust geospatial repartition by integrating historical data with LLM-generated candidates. |