Papers by Bandhav Veluri
AV-Dialog: Spoken Dialogue Models with Audio-Visual Input (2026.acl-long)
Copied to clipboard
| Challenge: | AV-Dialog uses audio and visual cues to track the target speaker, predict turn-taking, and generate coherent responses. |
| Approach: | They propose a multimodal dialog framework that uses both audio and visual cues to track the target speaker. |
| Outcome: | AV-Dialog outperforms audio-only models under interference, reducing transcription errors, improving turn-taking prediction and human-rated dialogue quality. |
Beyond Turn-Based Interfaces: Synchronous LLMs as Full-Duplex Dialogue Agents (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing spoken dialogue models are half-duplex in nature and require explicit prompting by the user or implicit tracking of interruption or silence events. |
| Approach: | They propose to integrate time information into Llama3-8b so that they run synchronously with the real-world clock. |
| Outcome: | The proposed model outperforms state-of-the-art in dialogue meaningfulness while maintaining naturalness. |