Papers by Reynold Bailey
Multimodal Modeling of Task-Mediated Confusion (2022.naacl-srw)
Copied to clipboard
| Challenge: | Existing models for confusion detection are under-explored, but they can be used to detect it computationally. |
| Approach: | They build upon prior work to develop models that detect confusion from three modalities: video, audio, and text. |
| Outcome: | The proposed models can detect confusion from facial expressions, prosody, and transcribed spoken language. |
MULTICOLLAB: A Multimodal Corpus of Dialogues for Analyzing Collaboration and Frustration in Language (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods to study complex emotions when a speaker collaborates with a partner are limited. |
| Approach: | They propose to fuse a multimodal dialogue resource with transcribed speech and eye gaze data to create a highly multimodal corpus. |
| Outcome: | The proposed model improves classification accuracy by 21% over baseline using sensor and speech data in 4.5 seconds. |