WavAlign: Enhancing Intelligence and Expressiveness in Spoken Dialogue Models via Adaptive Hybrid Post-Training (2026.findings-acl)
Copied to clipboard
Yifu Chen, Shengpeng Ji, Qian Chen, Tianle Liang, Yangzhuo Li, Ziqing Wang, Wen Wang, Jingyu Lu, Haoxiao Wang, Xueyi Pu, Fan Zhuo, Zhou Zhao
| Challenge: | End-to-end spoken dialogue models have higher potential ceiling in expressiveness and perceptual ability than cascaded systems. |
| Approach: | They propose a modality-aware adaptive post-training recipe that constrains preference updates to the semantic channel and improves acoustic behavior via explicit anchoring. |
| Outcome: | The proposed model improves speech quality and expressiveness across spoken dialogue benchmarks and architectures. |
Similar Papers
Human-centric dialog training via offline reinforcement learning (2020.emnlp-main)
Copied to clipboard
Natasha Jaques, Judy Hanwen Shen, Asma Ghandeharioun, Craig Ferguson, Agata Lapedriza, Noah Jones, Shixiang Gu, Rosalind Picard
| Challenge: | a novel offline RL method can train dialog models to produce better conversations without the risk of humans teaching it harmful chat behaviors. |
| Approach: | They develop offline reinforcement learning algorithms that use human feedback to train dialog models . they use language similarity, laughter, sentiment, and more to identify positive feedback . |
| Outcome: | The proposed method improves on existing methods with 80 users in an open-domain setting. |
Dual-Axis Generative Reward Model Toward Semantic and Turn-taking Robustness in Interactive Spoken Dialogue Models (2026.acl-long)
Copied to clipboard
Yifu Chen, Shengpeng Ji, Zhengqing Liu, Qian Chen, Wen Wang, Ziqing Wang, Yangzhuo Li, Tianle Liang, Zhou Zhao
| Challenge: | Reinforcement learning (RL) has improved text- and vision-language models, but its application in SDMs is hindered. |
| Approach: | They propose a dual-axis Generative Reward Model that provides semantic quality and interaction timing for SDMs. |
| Outcome: | The proposed model achieves state-of-the-art performance on interaction-quality assessment across a wide spectrum of datasets. |
Optimizing Conversational Quality in Spoken Dialogue Systems with Reinforcement Learning from AI Feedback (2026.findings-acl)
Copied to clipboard
Siddhant Arora, Jinchuan Tian, Jiatong Shi, Hayato Futami, Yosuke Kashiwagi, Emiru Tsunoo, Shinji Watanabe
| Challenge: | Existing studies on reinforcement learning from human or AI feedback have focused on semantic rewards at the utterance level. |
| Approach: | They propose a multi-reward RLAIF framework for speech-in/speech-out dialogue systems . they combine semantic, audio-quality, and emotion-consistency rewards . |
| Outcome: | The proposed framework improves speech-in/speech-out dialogue system quality . it combines semantic, audio-quality, and emotion-consistency rewards . the proposed framework is available to download from the cdc. |
Leveraging Implicit Feedback from Deployment Data in Dialogue (2024.eacl-short)
Copied to clipboard
| Challenge: | Xu et al., 2023) and Bai ed., 2019) use crowdworkers to collect signals from natural dialogue episodes. |
| Approach: | They use the publicly released BlenderBot deployment data to extract signals from conversations to implicitly measure the quality of a machine-generated utterance. |
| Outcome: | The proposed model improves over baseline models, but some proxy signals can lead to undesirable generations. |
SDiaReward: Modeling and Benchmarking Spoken Dialogue Rewards with Modality and Colloquialness (2026.acl-long)
Copied to clipboard
Jingyu Lu, Yuhan Wang, Fan Zhuo, Xize Cheng, Changhao Pan, Xueyi Pu, Yifu Chen, Chenyuhao Wen, Tianle Liang, Zhou Zhao
| Challenge: | SDiaReward is an end-to-end spoken dialogue system that integrates paralinguistic nuances and spontaneous nature of human conversation. |
| Approach: | They propose an end-to-end multi-turn reward model trained on SDiaReward-Dataset . it is a collection of episode-level preference pairs targeting modality and colloquiality gaps . |
| Outcome: | The proposed model outperforms general-purpose audio LLMs in episode-level evaluation. |
CHAI: A CHatbot AI for Task-Oriented Dialogue with Offline Reinforcement Learning (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to training dialogue agents are supervised learning, but this is prohibitively expensive and time-consuming. |
| Approach: | They propose offline reinforcement learning methods that can be used to train dialogue agents . offline reinforcement learn methods can be combined with language models to yield realistic dialogue agents. |
| Outcome: | The proposed method can be combined with language models to produce realistic dialogue agents . the results show that the offline method can achieve the goal of the proposed system . |
Plug-and-Play Conversational Models (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Large conversational models that generate coherent and fluent responses often require large dialogue datasets. |
| Approach: | They propose and evaluate plug-and-play methods for controllable response generation . they demonstrate a high degree of control over the generated conversational responses . |
| Outcome: | The proposed method does not require further computation at decoding time and does not need fine-tuning of a large language model. |
Rewarding What Matters: Step-by-Step Reinforcement Learning for Task-Oriented Dialogue (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing RL methods focus on generation tasks while neglecting dialogue state tracking (DST) for understanding. |
| Approach: | They propose a method that integrates RL into both understanding and generation tasks by introducing step-by-step rewards throughout the token generation. |
| Outcome: | The proposed approach achieves state-of-the-art results on three widely used datasets. |
Behavior-SD: Behaviorally Aware Spoken Dialogue Generation with Large Language Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Spoken dialogues lack explicit modeling of behavior traits that are often overlooked in language models . et al.: our work opens new possibilities for developing behaviorally-aware dialogue systems . |
| Approach: | They propose a large-scale dataset with over 100K spoken dialogues (2,164 hours) they propose BeDLM, the first dialogue model capable of generating natural conversations . |
| Outcome: | The proposed model outperforms baseline models in generating natural dialogues . the proposed model can generate natural conversations conditioned on behavioral and narrative contexts - a key feature of spoken language models . |
Rethinking Supervised Learning and Reinforcement Learning in Task-Oriented Dialogue Systems (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Dialogue policy learning for task-oriented dialogue systems has enjoyed great progress through using reinforcement learning methods. |
| Approach: | They propose a dialogue action decoder and a simulator-free adversarial learning method to improve dialogue agent performance without using reinforcement learning. |
| Outcome: | The proposed methods achieve more stable and higher performance with fewer efforts, such as the domain knowledge required to design a user simulator and the intractable parameter tuning in reinforcement learning. |