Introducing MULAI: A Multimodal Database of Laughter during Dyadic Interactions (2020.lrec-1)
Copied to clipboard
| Challenge: | Many theories exist about the origin and function of laughter, however, most theorists agree that it plays a major role in day to day social interactions between humans. |
| Approach: | They use the Multimodal Laughter during Interaction database to study laughter expressive patterns . they use acoustic laughter properties and annotated humour ratings to explore the link between humor and laughter properties. |
| Outcome: | The proposed database combines 601 laughs, 168 speech-laughs and 538 on- or offset respirations with other data rarely captured by other laughter databases. |
Similar Papers
When to Laugh and How Hard? A Multimodal Approach to Detecting Humor and Its Intensity (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods to generate humor using multimodal data are needed to study the role of humor in human social function. |
| Approach: | They propose a model that automatically detects humor in the Friends TV show using multimodal data and use prerecorded laughter as annotation as it marks humor. |
| Outcome: | The proposed model detects humor 78% of the time and how long the audience’s laughter reaction should last with a mean absolute error of 600 milliseconds. |
SMILE-Next: Teaching Large Language Models to Detect, Classify, and Reason about Laughter (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to understanding laughter or humor focus on narrowly defined tasks such as detecting humor and estimating humor intensity. |
| Approach: | They propose a dataset for real-world laughter understanding with multimodal textual representations and question–answer annotations. |
| Outcome: | The proposed framework outperforms baselines in three laughter-related tasks, showing that it is robust. |
MUCH: A Multimodal Corpus Construction for Conversational Humor Recognition Based on Chinese Sitcom (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing multimodal corpora for conversational humor are coarse-grained and insufficient to support the conversational comprehension task. |
| Approach: | They constructed a multimodal humor corpus based on a Chinese sitcom and used both unimodal and multimodal methods to test the corpus. |
| Outcome: | The proposed method outperforms unimodal and multimodal methods in the evaluation of a Chinese sitcom for conversational humor recognition. |
A Sentiment and Emotion Aware Multimodal Multiparty Humor Recognition in Multilingual Conversational Setting (2022.coling-1)
Copied to clipboard
| Challenge: | Humor is an essential aspect of daily conversation, and people try to provoke humor in their talks. |
| Approach: | They propose a multitask framework that annotates Hindi utterances with sentiment and emotion classes. |
| Outcome: | The proposed framework improves on the recently released Hindi Humor dataset . it takes sentiment and emotion into account to understand humor . |
UR-FUNNY: A Multimodal Language Dataset for Understanding Humor (D19-1)
Copied to clipboard
Md Kamrul Hasan, Wasifur Rahman, AmirAli Bagher Zadeh, Jianyuan Zhong, Md Iftekhar Tanveer, Louis-Philippe Morency, Mohammed (Ehsan) Hoque
| Challenge: | Humor is a unique and creative communicative behavior often displayed during social interactions. |
| Approach: | They present a dataset that allows to model multimodal language used in expressing humor using text, visual and acoustic communication. |
| Outcome: | The proposed framework opens the door to understanding multimodal language used in expressing humor. |
StandUp4AI: A New Multilingual Dataset for Humor Detection in Stand-up Comedy Videos (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a new multimodal dataset of stand-up comedies is proposed to improve humor detection . the dataset is the biggest available for this type of task, and the most diverse . |
| Approach: | They propose a method to enhance the automatic laughter detection based on Audio Speech Recognition errors. |
| Outcome: | The proposed method improves existing models of humor detection by using audio speech recognition errors. |
SMILE: Multimodal Dataset for Understanding Laughter in Video with Language Models (2024.findings-naacl)
Copied to clipboard
| Challenge: | Despite advances in artificial intelligence, building social intelligence remains a challenge. |
| Approach: | They propose a task to explain why people laugh in a video and a dataset to do this. |
| Outcome: | The proposed dataset generates plausible explanations for laughter in video and in-the-wild videos. |
Annotating Interruption in Dyadic Human Interaction (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing interruption and turn switch classification methods are not yet available. |
| Approach: | They propose a new interruption annotation schema that integrates existing interruption and turn switch classification methods to annotate different types of interruptions. |
| Outcome: | The proposed method can distinguish smooth turn exchange, backchannel and interruption (including interruption types) and to annotate dyadic conversation. |
The Spot the Difference corpus: a multi-modal corpus of spontaneous task oriented spoken interactions (L18-1)
Copied to clipboard
| Challenge: | a new corpus of task-oriented spontaneous dialogues is available for free . we study the structure of non task-orientated dialogues and how they evolve over time . |
| Approach: | They describe a corpus of 54 interactions between pairs of subjects interacting to find differences in two very similar scenes. |
| Outcome: | The proposed dataset contains 54 interactions between subjects to find differences in two very similar scenes. |
Construction and Analysis of a Multimodal Chat-talk Corpus for Dialog Systems Considering Interpersonal Closeness (2020.lrec-1)
Copied to clipboard
| Challenge: | a large-scale multimodal dialog corpus is needed to accelerate research on dialog systems that can handle social signals and verbal information. |
| Approach: | They construct a multimodal dialog corpus focusing on the relationship between speakers and 19 pairs of participants. |
| Outcome: | The proposed system is based on a multimodal dialog corpus of 19,303 utterances (10 hours) from 19 pairs of participants. |