Eliciting Implicit Acoustic Styles from Open-domain Instructions to Facilitate Fine-grained Controllable Generation of Speech (2025.emnlp-main)
Copied to clipboard
| Challenge: | Current work relies on pre-defined rules or templates to control the style of speech. |
| Approach: | They propose to use open-domain instructions to generate speech with the acoustic style that meets users’ needs based on their instructions. |
| Outcome: | The proposed model can be used to generate speech with the acoustic style that meets users’ needs based on open-domain instructions. |
Similar Papers
LLMs for Low Resource Languages in Multilingual, Multimodal and Dialectal Settings (2024.eacl-tutorials)
Copied to clipboard
| Challenge: | Recent advances in AI can be attributed to the remarkable performance of Large Language Models (LLMs) success of LLMs depends on specific training techniques, such as instruction tuning and prompting . |
| Approach: | They explore the capabilities of Large Language Models (LLMs) in various tasks and languages . they also examine their performance, fine-tuning, instructions tuning, and close vs. open models . |
| Outcome: | The proposed model can be used for speech and multimodal tasks across modalities, languages, and dialects. |
Towards Controllable Speech Synthesis in the Era of Large Language Models: A Systematic Survey (2025.emnlp-main)
Copied to clipboard
| Challenge: | Text-to-speech (TTS) has advanced from generating natural-sounding speech to enabling fine-grained control over speech attributes. |
| Approach: | They provide a review of controllable TTS methods from traditional control techniques to emerging approaches using natural language prompts. |
| Outcome: | The proposed methods are based on models, strategies, and features, and summarize challenges, datasets, and evaluations. |
Style Vectors for Steering Generative Large Language Models (2024.findings-eacl)
Copied to clipboard
Kai Konen, Sophie Jentzsch, Diaoulé Diallo, Peer Schütt, Oliver Bensch, Roxanne El Baff, Dominik Opitz, Tobias Hecking
| Challenge: | Large language models (LLMs) can be trained on vast corpora and can generate text in a nuanced and parameterisable way. |
| Approach: | They propose to add style vectors to the activations of hidden layers during text generation to steer output towards specific styles. |
| Outcome: | The proposed approach differs from prompt engineering in that it can be nuanced and parameterisable. |
Auto-Instruct: Automatic Instruction Generation and Ranking for Black-Box Language Models (2023.findings-emnlp)
Copied to clipboard
Zhihan Zhang, Shuohang Wang, Wenhao Yu, Yichong Xu, Dan Iter, Qingkai Zeng, Yang Liu, Chenguang Zhu, Meng Jiang
| Challenge: | Large language models can perform a wide range of tasks by following natural language instructions without task-specific fine-tuning. |
| Approach: | They propose a method to automatically improve the quality of LLM instructions . they leverage the generative ability of LMS to generate diverse candidate instructions based on a scoring model trained on 575 existing NLP tasks. |
| Outcome: | The proposed method surpasses human-written and LLM-generated instructions on 118 out-of-domain tasks. |
Toward Beginner-Friendly LLMs for Language Learning: Controlling Difficulty in Conversation (2026.findings-eacl)
Copied to clipboard
| Challenge: | Practicing conversations with large language models is a promising alternative to traditional in-person language learning. |
| Approach: | They propose a new token-level evaluation metric, Token Miss Rate, that measures the proportion of incomprehensible tokens per utterance and correlates strongly with human judgments. |
| Outcome: | The proposed methods improve comprehensibility for beginner speakers from 39.4% to 83.3%, compared with prompting alone and a token-level evaluation metric, Token Miss Rate (TMR). |
Controlled Language Generation for Language Learning Items (2022.emnlp-industry)
Copied to clipboard
| Challenge: | Recent advances in pre-trained language models have resulted in success in generating fluent English text. |
| Approach: | They propose to employ natural language generation to rapidly generate English language items . they experiment with deep pretrained models and develop methods for controlling items for factors relevant in language learning . |
| Outcome: | The proposed framework shows high grammatically scores for all models and higher complexity over baseline models. |
Seed-Free Synthetic Data Generation Framework for Instruction-Tuning LLMs: A Case Study in Thai (2024.acl-srw)
Copied to clipboard
| Challenge: | Xue et al., 2024) have demonstrated that large language models can perform at human level across multitudes of tasks and domains. |
| Approach: | They propose a seed-free framework for generating synthetic instruction-tuning data that incorporates fluency, diversity, and cultural context. |
| Outcome: | The proposed framework achieves competitive performance using only 5,000 instructions compared to state-of-the-art Thai LLMs trained on hundreds of thousands of instructions. |
CodecLM: Aligning Language Models with Tailored Synthetic Data (2024.findings-naacl)
Copied to clipboard
Zifeng Wang, Chun-Liang Li, Vincent Perot, Long Le, Jin Miao, Zizhao Zhang, Chen-Yu Lee, Tomas Pfister
| Challenge: | Recent work on generating diverse instructions and applying LLM to increase instruction complexity neglects downstream use cases. |
| Approach: | They propose a framework for generating high-quality synthetic data for LLM alignment with different downstream instruction distributions and LLMs. |
| Outcome: | Experiments on four open-domain instruction using the proposed framework validate the effectiveness of CodecLM over the current state-of-the-art. |
Ensemble-Instruct: Instruction Tuning Data Generation with a Heterogeneous Mixture of LMs (2023.findings-emnlp)
Copied to clipboard
Young-Suk Lee, Md Sultan, Yousef El-Kurdi, Tahira Naseem, Asim Munawar, Radu Florian, Salim Roukos, Ramón Astudillo
| Challenge: | Empirical studies with different instruction-tuned LMs show that our proposed method yields higher-quality instruction tuning data than Self-Instruct. |
| Approach: | They propose to use in-context learning techniques to train strong conversational agents . they propose to categorize and simplify ICL templates to make prompt learning easier . |
| Outcome: | Empirical results show that the proposed method yields higher-quality instruction tuning data than Self-Instruct and improves performance of both vanilla and instruction-tuned LMs. |
A Rigorous Evaluation of LLM Data Generation Strategies for Low-Resource Languages (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly used to generate synthetic textual data for training smaller specialized models. |
| Approach: | They evaluate the performance of large language models and their generation strategies in 11 different languages using 3 NLP tasks and 4 open-source LLMs. |
| Outcome: | The proposed generation strategies and their combinations yield strong results across 11 languages, including several extremely low-resource ones. |