SMILE: Multimodal Dataset for Understanding Laughter in Video with Language Models (2024.findings-naacl)
Copied to clipboard
| Challenge: | Despite advances in artificial intelligence, building social intelligence remains a challenge. |
| Approach: | They propose a task to explain why people laugh in a video and a dataset to do this. |
| Outcome: | The proposed dataset generates plausible explanations for laughter in video and in-the-wild videos. |
Similar Papers
SMILE-Next: Teaching Large Language Models to Detect, Classify, and Reason about Laughter (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to understanding laughter or humor focus on narrowly defined tasks such as detecting humor and estimating humor intensity. |
| Approach: | They propose a dataset for real-world laughter understanding with multimodal textual representations and question–answer annotations. |
| Outcome: | The proposed framework outperforms baselines in three laughter-related tasks, showing that it is robust. |
Multimodal and Multilingual Laughter Detection in Stand-Up Comedy Videos (2024.lrec-main)
Copied to clipboard
| Challenge: | Using TED talks, we use laughter detection software to capture humor in the sitcom genre. |
| Approach: | They develop a multimodal multilingual dataset in Russian and English with a particular emphasis on laughter detection techniques. |
| Outcome: | The proposed model outperforms peak detection and machine learning, while the latter shows promise and warrants further study. |
Can Language Models Laugh at YouTube Short-form Videos? (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing datasets that focus on verbal cues and focus on short-form funny videos focus on focusing on verbs and visual cue. |
| Approach: | They curate a user-generated dataset of 10K multimodal funny videos from YouTube and annotate each video with timestamps and explanations for funny moments. |
| Outcome: | The proposed dataset improves the ability of large language models to understand humor. |
StandUp4AI: A New Multilingual Dataset for Humor Detection in Stand-up Comedy Videos (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a new multimodal dataset of stand-up comedies is proposed to improve humor detection . the dataset is the biggest available for this type of task, and the most diverse . |
| Approach: | They propose a method to enhance the automatic laughter detection based on Audio Speech Recognition errors. |
| Outcome: | The proposed method improves existing models of humor detection by using audio speech recognition errors. |
UR-FUNNY: A Multimodal Language Dataset for Understanding Humor (D19-1)
Copied to clipboard
Md Kamrul Hasan, Wasifur Rahman, AmirAli Bagher Zadeh, Jianyuan Zhong, Md Iftekhar Tanveer, Louis-Philippe Morency, Mohammed (Ehsan) Hoque
| Challenge: | Humor is a unique and creative communicative behavior often displayed during social interactions. |
| Approach: | They present a dataset that allows to model multimodal language used in expressing humor using text, visual and acoustic communication. |
| Outcome: | The proposed framework opens the door to understanding multimodal language used in expressing humor. |
Do Androids Laugh at Electric Sheep? Humor “Understanding” Benchmarks from The New Yorker Caption Contest (2023.acl-long)
Copied to clipboard
Jack Hessel, Ana Marasovic, Jena D. Hwang, Lillian Lee, Jeff Da, Rowan Zellers, Robert Mankoff, Yejin Choi
| Challenge: | Large neural networks can generate jokes, but do they really “understand” humor? a new challenge challenges AI models to match a joke to a cartoon, identify a winning caption, and explain why a winner is funny. |
| Approach: | They propose three tasks based on the New Yorker Cartoon Caption Contest . they aim to match a joke to a cartoon, identify a winning caption and explain why it's funny . |
| Outcome: | The proposed tasks are based on the New Yorker Cartoon Caption Contest . they include matching a joke to a cartoon, identifying a winning caption, and explaining why a funny caption is funny. |
v-HUB: A Benchmark for Video Humor Understanding from Vision and Sound (2026.acl-long)
Copied to clipboard
Zhengpeng Shi, Yanpeng Zhao, Jianqun Zhou, Yuxuan Wang, Qinrong Cui, Wei Bi, Song-Chun Zhu, Bo Zhao, Zilong Zheng
| Challenge: | Humor enriches our daily lives and appears in many forms, from jokes and cartoons to comedies and viral videos. |
| Approach: | They introduce a video humor understanding benchmark to test their ability to understand humor from visual cues. |
| Outcome: | The proposed video humor understanding benchmark is based on a collection of short videos . it features rich annotations and a study of environmental sound that can enhance humor . |
ACQUIRED: A Dataset for Answering Counterfactual Questions In Real-Life Videos (2023.emnlp-main)
Copied to clipboard
Te-Lin Wu, Zi-Yi Dou, Qingyuan Hu, Yu Hou, Nischal Chandra, Marjorie Freedman, Ralph Weischedel, Nanyun Peng
| Challenge: | despite its importance, there are few datasets that cover multimodal counterfactual reasoning . a dataset focusing on this area is limited because of its limited coverage over synthetic environments . |
| Approach: | They develop a video question answering dataset that provides questions on multimodal reasoning . they ask questions about counterfactual hypotheses over visual events . |
| Outcome: | The proposed dataset shows a significant performance gap between models and humans . it provides questions that span physical, social, and temporal dimensions . |
Large Language Models Are Natural Video Popularity Predictors (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) can better capture cultural and social factors such as viewing intensity and geographic spread of video content. |
| Approach: | They propose to use Large Language Models to capture cultural and social factors that influence video popularity and generate interpretable, attribute-based explanations. |
| Outcome: | The proposed model captures both engagement intensity and geographic spread on 13,639 popular videos, while the neural network's predictions reach 82% without fine-tuning. |
Large Dataset and Language Model Fun-Tuning for Humor Recognition (P19-1)
Copied to clipboard
| Challenge: | Humor recognition datasets contain only English texts and focus on puns. |
| Approach: | They collected a dataset of jokes and funny dialogues in Russian and complemented them carefully with unfunny texts with similar lexical properties. |
| Outcome: | The proposed method is based on the universal language model finetuning and has an F1 score of 0.91 on a test set. |