Papers by Berrak Sisman
Discovering and Causally Validating Emotion-Sensitive Neurons in Large Audio-Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Emotion is a central dimension of spoken communication, yet we lack a mechanistic account of how LALMs encode it internally. |
| Approach: | They propose to use emotion-sensitive neurons in large audio-language models to study their interpretations. |
| Outcome: | The proposed models show that they can be used to make decisions on emotion . the results show that the ESNs exhibit non-uniform clustering with partial cross-dataset transfer . |
Multimodal Fine-grained Context Interaction Graph Modeling for Conversational Speech Synthesis (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods overlook the fine-grained semantic and prosodic interaction modeling at the word level. |
| Approach: | They propose a novel approach to generate conversational prosody by understanding multimodal dialogue history (MDH) using fine-grained semantic and prosodic interaction modeling, they construct specialized multimodal fine-grain dialogue interaction graphs that encode interaction between word-level semantics and prosody. |
| Outcome: | The proposed system outperforms baseline models in terms of prosodic expressiveness. |