Beyond the Turn-Based Game: Enabling Real-Time Conversations with Duplex Models (2024.emnlp-main)
Copied to clipboard
Xinrong Zhang, Yingfa Chen, Shengding Hu, Xu Han, Zihang Xu, Yuanwei Xu, Weilin Zhao, Maosong Sun, Zhiyuan Liu
| Challenge: | Large language models (LLMs) are increasingly permeating daily lives and require real-time interactions that mirror human conversations. |
| Approach: | They propose to use time-division-multiplexing to process queries and responses pseudo-simultaneously. |
| Outcome: | The proposed model can listen to users while generating output and adjust to provide instant feedback. |
Similar Papers
Beyond Turn-Based Interfaces: Synchronous LLMs as Full-Duplex Dialogue Agents (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing spoken dialogue models are half-duplex in nature and require explicit prompting by the user or implicit tracking of interruption or silence events. |
| Approach: | They propose to integrate time information into Llama3-8b so that they run synchronously with the real-world clock. |
| Outcome: | The proposed model outperforms state-of-the-art in dialogue meaningfulness while maintaining naturalness. |
Sommelier: Scalable Open Multi-turn Audio Pre-processing for Full-duplex Speech Language Models (2026.acl-industry)
Copied to clipboard
| Challenge: | Existing high-quality conversational data is limited for full-duplex models . overlapping and backchanneling are a challenge for most systems . |
| Approach: | They propose a robust and scalable open-source data processing pipeline for full-duplex models. |
| Outcome: | The proposed pipeline can listen and speak simultaneously, supporting more fluid and human-like interaction. |
Fairness Evaluation and Inference Level Mitigation in LLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models display undesirable behaviors embedded in their internal representations, undermining fairness, inconsistency drift, and the propagation of unwanted patterns during extended dialogues. |
| Approach: | They propose a pruning-based framework that detects context-aware neuron activations and applies adaptive masking to modulate their influence during generation. |
| Outcome: | The proposed framework detects context-aware neuron activations and applies adaptive masking to modulate their influence during generation. |
BotChat: Evaluating LLMs’ Capabilities of Having Multi-Turn Dialogues (2024.findings-naacl)
Copied to clipboard
Haodong Duan, Jueqi Wei, Chonghua Wang, Hongwei Liu, Yixiao Fang, Songyang Zhang, Dahua Lin, Kai Chen
| Challenge: | Modern Large Language Models (LLMs) facilitate high-quality, multi-turn dialogues with humans, but human-based evaluation of such a capability requires substantial manual effort. |
| Approach: | They propose to evaluate LLMs' ability to emulate human-like, multi-turn conversations using an LLM-centric approach. |
| Outcome: | The proposed model emulates human-like, multi-turn conversations using an LLM-centric approach. |
MTR-DuplexBench: Towards a Comprehensive Evaluation of Multi-Round Conversations for Full-Duplex Speech Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks focus on evaluating single-round interactions, neglecting other critical aspects. |
| Approach: | They propose a benchmark to evaluate full-duplex speech language models in multi-round settings . they segment continuous full-dual dialogues into discrete turns for evaluation . |
| Outcome: | The proposed benchmark compared full-duplex speech language models with full-dual speech models . the results show that the models perform better in multi-round settings than standard models compared to benchmarks . |
Time to Talk: LLM Agents for Asynchronous Group Communication in Mafia Games (2025.findings-emnlp)
Copied to clipboard
| Challenge: | LLMs are used in synchronous communication, where a human user and a model communicate in alternating turns. |
| Approach: | They develop an adaptive asynchronous LLM agent consisting of two modules that decide what to say and a scheduler that decides when to say it. |
| Outcome: | The proposed agent performs on par with human players in online Mafia games and in its ability to blend in with the other human players. |
MT-Eval: A Multi-Turn Capabilities Evaluation Benchmark for Large Language Models (2024.emnlp-main)
Copied to clipboard
Wai-Chung Kwan, Xingshan Zeng, Yuxin Jiang, Yufei Wang, Liangyou Li, Lifeng Shang, Xin Jiang, Qun Liu, Kam-Fai Wong
| Challenge: | Existing evaluation frameworks focus on single-turn evaluations, overlooking the models’ capabilities in multi-turn interactions. |
| Approach: | They propose a benchmark to evaluate the multi-turn conversational abilities of large language models (LLMs) by analyzing human-LLM conversations and constructing multi-turned queries for each category using GPT-4. |
| Outcome: | The proposed model outperforms open-source models in multi-turn tasks while retaining and recalling historical information. |
F-Actor: Controllable Conversational Behavior in Full-Duplex Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Current spoken conversational systems lack customization capabilities, limiting their naturalness and usability. |
| Approach: | They propose an instruction-following full-duplex conversational speech model that can be trained efficiently under typical academic resource constraints. |
| Outcome: | The proposed model requires just 2,000 hours of data to be trained under typical academic resource constraints. |
Rethinking Text-to-SQL: Dynamic Multi-turn SQL Interaction for Real-world Database Exploration (2026.findings-acl)
Copied to clipboard
Linzhuang Sun, Tianyu Guo, Hao Liang, Ruitong Liu, Yuying Li, Qifeng Cai, Jingxuan Wei, Yuchen Wu, Bihui Yu, Xiangxiang Zhang, Wentao Zhang, Bin Cui
| Challenge: | Structured Query Language (SQL) is the cornerstone for data-driven decision-making. |
| Approach: | They propose a benchmark to rigorously evaluate Large Language Models within a dynamic interaction framework. |
| Outcome: | The proposed benchmark aims to rigorously evaluate LLMs within a dynamic interaction framework. |
A Survey on LLM-based Conversational User Simulation (2026.eacl-long)
Copied to clipboard
Bo Ni, Yu Wang, Leyao Wang, Branislav Kveton, Franck Dernoncourt, Yu Xia, Hongjie Chen, Reuben Luera, Samyadeep Basu, Subhojyoti Mukherjee, Puneet Mathur, Nesreen K. Ahmed, Junda Wu, Li Li, Huixin Zhang, Ruiyi Zhang, Tong Yu, Sungchul Kim, Jiuxiang Gu, Zhengzhong Tu, Alexa Siu, Zichao Wang, Seunghyun Yoon, Nedim Lipka, Namyong Park, Zihao Lin, Trung Bui, Yue Zhao, Tyler Derr, Ryan A. Rossi
| Challenge: | Recent advances in large language models (LLMs) have enabled high-fidelity generation of synthetic user conversation. |
| Approach: | They propose a taxonomy covering user granularity and simulation objectives . they analyze core techniques and evaluation methodologies to help them understand the latest developments . |
| Outcome: | The proposed model enables high-fidelity generation of synthetic user conversation. |