Papers by Erik Ekstedt
Multilingual Turn-taking Prediction Using Voice Activity Projection (2024.lrec-main)
Copied to clipboard
| Challenge: | a monolingual model does not make good predictions when applied to other languages, but a multilingual model is able to discern the language of the input signal. |
| Approach: | They propose to use a multilingual voice activity projection model to predict voice activities of spoken dialogue participants in English, Mandarin, and Japanese data. |
| Outcome: | The proposed model predicts the upcoming voice activities of participants in dyadic dialogue on multilingual data, encompassing English, Mandarin, and Japanese. |
Response-conditioned Turn-taking Prediction (2023.findings-acl)
Copied to clipboard
| Challenge: | Traditionally, turn-taking is done using a simple silence threshold, but more modern approaches use cues known to be important in human-human turn-shifts. |
| Approach: | They propose a turn-taking and response-ranking model that conditions the end-of-turn prediction on conversation history and what the next speaker wants to say. |
| Outcome: | The proposed model outperforms the baseline model in a variety of metrics. |
TurnGPT: a Transformer-based Language Model for Predicting Turn-taking in Spoken Dialog (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Syntactic and pragmatic completeness is important for turn-taking prediction, but so far machine learning models of turn- taking have used such linguistic information in a limited way. |
| Approach: | They introduce a transformer-based language model for predicting turn-shifts in spoken dialog and evaluate it against a variety of datasets. |
| Outcome: | The proposed model outperforms two baseline models on spoken and written dialog datasets and can detect and project turn completions. |