Creating a Lens of Chinese Culture: A Multimodal Dataset for Chinese Pun Rebus Art Understanding (2025.findings-acl)
Copied to clipboard
Tuo Zhang, Tiantian Feng, Yibin Ni, Mengqin Cao, Ruying Liu, Kiana Avestimehr, Katharine Butler, Yanjun Weng, Mi Zhang, Shrikanth Narayanan, Salman Avestimehr
| Challenge: | a new study examines the performance of large vision-language models in understanding art . the Pun Rebus Art Dataset is a multimodal dataset for art understanding rooted in traditional Chinese culture . |
| Approach: | They propose a multimodal dataset for art understanding deeply rooted in traditional Chinese culture . they aim to facilitate the development of VLMs that can better understand culturally specific content . |
| Outcome: | The proposed dataset shows that state-of-the-art VLMs struggle with these tasks . the data will facilitate the development of VLM models that can better understand culturally specific content . |
Similar Papers
PunMemeCN: A Benchmark to Explore Vision-Language Models’ Understanding of Chinese Pun Memes (2025.emnlp-main)
Copied to clipboard
| Challenge: | Pun memes combine wordplay with visual elements to create humor, irony, or other rhetorical effects. |
| Approach: | They propose a benchmark to assess Chinese pun memes' processing capabilities across three progressive tasks: pun meme detection, sentiment analysis, and chat-driven meme response. |
| Outcome: | The proposed model can detect pun memes, analyze sentiments, and respond to chats, while ignoring homophone wordplay. |
Can MLLMs Understand the Deep Implication Behind Chinese Images? (2025.acl-long)
Copied to clipboard
Chenhao Zhang, Xi Feng, Yuelin Bai, Xeron Du, Jinchang Hou, Kaixin Deng, Guangzeng Han, Qinrui Li, Bingli Wang, Jiaheng Liu, Xingwei Qu, Yifei Zhang, Qixuan Zhao, Yiming Liang, Ziqiang Liu, Feiteng Fang, Min Yang, Wenhao Huang, Chenghua Lin, Ge Zhang, Shiwen Ni
| Challenge: | MLLMs perform poorly on traditional culture images, indicating limitations in understanding high-level semantics and lacking a deep knowledge base of Chinese traditional culture. |
| Approach: | They propose to use Chinese images to assess MLLMs' higher-order perception and understanding of Chinese visual content. |
| Outcome: | The proposed model incorporates images that represent Chinese traditional culture, such as famous Chinese traditional paintings, to ensure the authenticity of the Chinese context. |
“I See What You Did There”: Can Large Vision-Language Models Understand Multimodal Puns? (2026.acl-long)
Copied to clipboard
Naen Xu, Jiayi Sheng, Changjiang Li, Chunyi Zhou, Yuyuan Li, Tianyu Du, Jun Wang, Zhihui Fu, Jinbao Li, Shouling Ji
| Challenge: | Puns are a common form of rhetorical wordplay that exploits polysemy and phonetic similarity to create humor. |
| Approach: | They propose a multimodal pun generation pipeline and a model to evaluate their understanding of puns. |
| Outcome: | The proposed benchmark improves the understanding of multimodal puns by 16.5% in the F1 test. |
RealBench: A Chinese Multi-image Understanding Benchmark Close to Real-world Scenarios (2025.findings-emnlp)
Copied to clipboard
Fei Zhao, Chengqiang Lu, Yufan Shen, Qimeng Wang, Yicheng Qian, Haoxin Zhang, Yan Gao, null Yiwu, Yao Hu, Zhen Wu, Shangyu Xing, Xinyu Dai
| Challenge: | RealBench is the first Chinese multimodal multi-image dataset . the dataset contains 9393 samples and 69910 images . |
| Approach: | They propose to create a Chinese multimodal multi-image dataset using 21 models . they use closed-source models that support multi-inputs as well as open-source visual and video models a . |
| Outcome: | The first Chinese multimodal multi-image dataset contains 9393 samples and 69910 images. |
A Structured Framework for Evaluating and Enhancing Interpretive Capabilities of Multimodal LLMs in Culturally Situated Tasks (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Using a zero-shot classification model, we extracted multi-dimensional evaluative features from human expert critiques and used them to evaluate selected VLMs such as Llama, Qwen, or Gemini. |
| Approach: | They constructed a quantitative framework for Chinese painting critique by extracting multi-dimensional evaluative features from human expert critiques using a zero-shot classification model. |
| Outcome: | The framework was constructed by extracting features from human critiques using a zero-shot classification model. |
FoodieQA: A Multimodal Dataset for Fine-Grained Understanding of Chinese Food Culture (2024.emnlp-main)
Copied to clipboard
Wenyan Li, Crystina Zhang, Jiaang Li, Qiwei Peng, Raphael Tang, Li Zhou, Weijia Zhang, Guimin Hu, Yifei Yuan, Anders Søgaard, Daniel Hershcovich, Desmond Elliott
| Challenge: | FoodieQA is a manually curated, fine-grained image-text dataset capturing the intricate features of food cultures across various regions in China. |
| Approach: | They evaluate vision–language Models and large language models on unseen food images and corresponding questions. |
| Outcome: | The proposed dataset evaluates vision–language Models and large language models on unseen food images and corresponding questions. |
Pun2Pun: Benchmarking LLMs on Textual-Visual Chinese-English Pun Translation via Pragmatics Model and Linguistic Reasoning (2025.acl-srw)
Copied to clipboard
| Challenge: | Current approaches resort to suboptimal compromises and computational methods remain inadequate for translation. |
| Approach: | They propose a Constant-Variable Optimization (CVO) model for translation strategy and an Ovl metric for translation quality assessment that adapts to Chinese and English. |
| Outcome: | The proposed model improves performance on textual and visual puns while maintaining linguistic mechanisms and humorous effects. |
Hanfu-Bench: A Multimodal Benchmark on Cross-Temporal Cultural Understanding and Transcreation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on cultural understanding with vision-language models primarily emphasize geographic diversity, often overlooking the critical temporal dimensions. |
| Approach: | They propose a multimodal vision-language model that examines temporal features and cultural image transcreation. |
| Outcome: | The novel model performs better than non-experts on visual cutural understanding but falls short to human experts on cultural image transcreation task. |
Cultural Bias Matters: A Cross-Cultural Benchmark Dataset and Sentiment-Enriched Model for Understanding Multimodal Metaphors (2025.acl-long)
Copied to clipboard
| Challenge: | Metaphors are pervasive in communication, making them crucial for natural language processing. |
| Approach: | They propose a multicultural multimodal metaphor dataset designed for cross-cultural studies of metaphor in Chinese and English. |
| Outcome: | The proposed model improves metaphor comprehension across cultural backgrounds and cultural domains. |
Concept-pedia: a Wide-coverage Semantically-annotated Multimodal Dataset (2025.emnlp-main)
Copied to clipboard
| Challenge: | Current evaluations for Vision-language Models remain heavily anchored to ImageNet . |
| Approach: | They propose a large-scale semantically-annotated multimodal resource that extends the range of visual concepts, including diverse abstract categories. |
| Outcome: | The proposed model expands the range of visual concepts, including diverse abstract categories. |