| Challenge: | Injecting fillers into spoken dialogue systems has a rich history of study . ambiguity of filler occurrence and inter-speaker difference make modeling and evaluation difficult. |
| Approach: | They propose an objective score for filler insertion using sampling-based sampling . they build three models trained on two single-speaker spontaneous corpora and evaluate them with FPP and perceptual tests. |
| Outcome: | The proposed model is useful in analysis but does not correlate well with perceptual MOS. |
Similar Papers
FillerSpeech: Towards Human-Like Text-to-Speech Synthesis with Filler Insertion and Filler Style Control (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in speech synthesis have improved audio quality and pronunciation . fillers are an integral part of natural human conversation, but achieving human-like conversational speech remains a challenge. |
| Approach: | They propose a speech synthesis framework that enables natural filler insertion and style control . they propose 'filler-inclusive' speech data that includes fillers with pitch and duration information . |
| Outcome: | The proposed framework enables natural filler insertion and style control . the proposed framework is validated and can be used to predict filler style . |
Augmented Prompt Selection for Evaluation of Spontaneous Speech Synthesis (2020.lrec-1)
Copied to clipboard
| Challenge: | Spontaneous speech is unscripted and created on the fly by the speaker, whereas read speech is pre-planned. |
| Approach: | They propose a tool that allows developers to select a varied, representative set of utterances from a spoken genre to be used for evaluation of TTS for a given domain. |
| Outcome: | The proposed tool can be used to evaluate TTS for a given domain using visualisation and tree-based algorithm. |
The importance of fillers for text representations of speech transcripts (2020.emnlp-main)
Copied to clipboard
| Challenge: | Fillers are a type of disfluency that can be a sound ("um" or "uh") filling a pause in an utterance or conversation. |
| Approach: | They propose to represent fillers with deep contextualised embeddings to improve modelling of spoken language and two downstream tasks . |
| Outcome: | The proposed representations improve modelling of spoken language and two downstream tasks, predicting a speaker’s stance and expressed confidence. |
Evaluating Text-to-Speech Synthesis from a Large Discrete Token-based Speech Language Model (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent advances in generative language modeling applied to discrete speech tokens presented a new avenue for text-to-speech (TTS) synthesis. |
| Approach: | They propose to use generative language modeling to generate text-to-speech (TTS) outputs by a discrete token-based model. |
| Outcome: | The proposed model is rated higher in naturalness and context appropriateness in listening tests compared to a conventional TTS. |
InSerter: Speech Instruction Following with Unsupervised Interleaved Pre-training (2025.acl-long)
Copied to clipboard
Dingdong Wang, Jin Xu, Ruihang Chu, Zhifang Guo, Xiong Wang, Jincenzi Wu, Dongchao Yang, Shengpeng Ji, Junyang Lin
| Challenge: | Recent advances in speech large language models exhibit suboptimal performance in adhering to speech instructions. |
| Approach: | They propose a method to pre-train large-scale unsupervised speech-text sequences . they use text-to-speech conversion to generate textual continuations corresponding to provided speech segments . |
| Outcome: | The proposed model achieves superior or competitive results across diverse speech processing tasks. |
CM-TTS: Enhancing Real Time Text-to-Speech Synthesis Efficiency through Weighted Samplers and Consistency Models (2024.findings-naacl)
Copied to clipboard
| Challenge: | Neural Text-to-Speech systems are a promising approach for high-fidelity speech synthesis . but the efficiency of multi-step sampling in Diffusion Models presents challenges . |
| Approach: | They propose a novel architecture grounded in consistency models to improve model convergence. |
| Outcome: | The proposed architecture achieves top-quality speech synthesis in fewer steps without adversarial training or pre-trained model dependencies. |
On the Fallacy of Global Token Perplexity in Spoken Language Model Evaluation (2026.findings-acl)
Copied to clipboard
Chan-Jan Hsu, Liang-Hsuan Tseng, Yi-Cheng Lin, Yen-Chun Kuo, Ju-Chieh Chou, Kai-Wei Chang, Hung-yi Lee, Carlos Busso
| Challenge: | Generative spoken language models are often evaluated using global token perplexity, which overlooks fundamental differences between speech and text modalities. |
| Approach: | They propose a variety of likelihood- and generative-based evaluation methods that serve in place of naive global token perplexity. |
| Outcome: | The proposed evaluations more faithfully reflect perceived generation quality, as evidenced by stronger correlations with human-rated mean opinion scores (MOS). |
F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching (2025.acl-long)
Copied to clipboard
| Challenge: | Recent research in Text-to-Speech (TTS) has experienced great advancement . current models can synthesize speech for any given text and mimic the speaker of audio prompt. |
| Approach: | They propose a fully non-autoregressive text-to-speech system based on flow matching with Diffusion Transformer (DiT) without complex designs such as duration model, text encoder, and phoneme alignment, the text input is simply padded with filler tokens to the same length as input speech, and then denoising is performed for speech generation. |
| Outcome: | The proposed system achieves an inference RTF of 0.15, which is greatly improved compared to state-of-the-art diffusion-based models. |
Integrating Disfluency-based and Prosodic Features with Acoustics in Automatic Fluency Evaluation of Spontaneous Speech (2020.lrec-1)
Copied to clipboard
| Challenge: | acoustics, prosody, and disfluency-based features are used to evaluate fluent/disfluent speech . filling pauses and word fragments are used for automatic fluency evaluation . |
| Approach: | They integrate acoustics, prosody, and disfluency-based features into an automatic fluency evaluation task. |
| Outcome: | The proposed model improves when integrated with prosodic features, but not when disfluent speech is detected. |
The Role of Creaky Voice in Turn Taking and the Perception of Speaker Stance: Experiments Using Controllable TTS (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent advances in spontaneous text-to-speech (TTS) have enabled the realistic generation of creaky voice, a voice quality known for its diverse pragmatic and paralinguistic functions. |
| Approach: | They used a creaky voice detection tool and a neural TTS engine to control creaky phonation in a spontaneous speech corpus to investigate the effect of creaky voices on perceived certainty, valence, sarcasm, and turn finality. |
| Outcome: | The proposed model enables the realistic synthesis of creaky voice in perceptual tests without formal training. |