Papers by Jonah Luby
WhiSPA: Semantically and Psychologically Aligned Whisper with Self-Supervised Contrastive and Student-Teacher Learning (2025.acl-long)
Copied to clipboard
Rajath Rao, Adithya V Ganesan, Oscar Kjell, Jonah Luby, Akshay Raghavan, Scott M. Feltman, Whitney Ringwald, Ryan L. Boyd, Benjamin J. Luft, Camilo J. Ruggero, Neville Ryant, Roman Kotov, H. Schwartz
| Challenge: | Current speech encoding pipelines rely on an additional text-based LM to get robust representations of human communication, even though speech-to-text models often have a LM within. |
| Approach: | They propose to align Whisper's latent space with semantic representations from a text autoencoder and lexically derived embeddings of basic psychological dimensions: emotion and personality. |
| Outcome: | The proposed approach surpasses current speech encoders over self-supervised affective tasks and downstream psychological tasks, achieving an error reduction of 73.4% and 83.8%, respectively. |
Transfer and Active Learning for Dissonance Detection: Addressing the Rare-Class Challenge (2023.acl-long)
Copied to clipboard
Vasudha Varadarajan, Swanie Juhng, Syeda Mahwish, Xiaoran Liu, Jonah Luby, Christian Luhmann, H. Andrew Schwartz
| Challenge: | Active learning has been proposed to alleviate data acquisition challenges for rare-class tasks when the class label is very infrequent (e.g., 5% of samples). |
| Approach: | They propose to use transformers to train models on closely related tasks and evaluate acquisition strategies, including a proposed probability-of-rare-class approach to dissonance detection. |
| Outcome: | The proposed method improves model accuracy while iterative transfer-learning does not improve cold-start performance. |