Susu Box or Piggy Bank: Assessing Cultural Commonsense Knowledge between Ghana and the US (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent work has highlighted the culturally-contingent nature of commonsense knowledge . a multi-stage process is used to evaluate the commonsence of English LLMs . |
| Approach: | They propose a test set of 525 multiple-choice questions to evaluate commonsense knowledge of English LLMs in Ghana and the u.s. They use existing commonsensible datasets to rewrite them in a multi-stage process. |
| Outcome: | The proposed model improves on the culturally-contingent commonsense knowledge of English LLMs in Ghana and the United States. |
Similar Papers
Cross-Cultural Transfer of Commonsense Reasoning in LLMs: Evidence from the Arab World (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study examined the potential for cross-cultural transfer of commonsense reasoning . merely 12 culture-specific examples from one country can improve performance in others by 10% on average . |
| Approach: | They evaluate cross-cultural transfer of commonsense reasoning within the arab world . they use in-context learning and demonstration-based reinforcement to evaluate alignment methods . |
| Outcome: | The proposed model can improve performance in cultures with cultural similarities in the Arab world by 10% on average. |
MAKIEval: A Multilingual Automatic WiKidata-based Framework for Cultural Awareness Evaluation for LLMs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) are used globally across many languages, but their English-centric pretraining raises concerns about cross-lingual disparities for cultural awareness . |
| Approach: | They introduce an automatic multilingual framework for evaluating cultural awareness in large language models across languages, regions, and topics. |
| Outcome: | The framework evaluates open-ended text generation, capturing how models express culturally grounded knowledge in natural language. |
LLMs as Cultural Archives: Cultural Commonsense Knowledge Graph Extraction (2026.eacl-long)
Copied to clipboard
| Challenge: | Large language models encode rich cultural knowledge, but it remains mostly implicit and unstructured, limiting its interpretability and use. |
| Approach: | They propose an iterative framework for constructing a Cultural Commonsense Knowledge Graph using a prompt-based framework. |
| Outcome: | The proposed framework improves cultural reasoning and story generation on non-English cultures. |
Common to Whom? Regional Cultural Commonsense and LLM Bias in India (2026.acl-long)
Copied to clipboard
Sangmitra Madhusudan, Trush Shashank More, Steph Buongiorno, Renata Dividino, Jad Kabbara, Ali Emami
| Challenge: | Existing cultural commonsense benchmarks treat nations as monolithic, assuming uniform practices within national boundaries. |
| Approach: | They evaluate eight state-of-the-art LLMs and find two critical gaps . commonsense knowledge is fundamentally long-tailed, with most facts rare in training data . |
| Outcome: | The proposed model achieves only 13.4%–20.9% accuracy on region-specific questions and exhibits geographic bias over-selecting Central and North India as the "default" while under-representing East and West. |
Can LLM Generate Culturally Relevant Commonsense QA Data? Case Study in Indonesian and Sundanese (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly being used to generate synthetic data for training and evaluating models. |
| Approach: | They investigate the effectiveness of using Large Language Models to generate culturally relevant commonsense QA datasets for Indonesian and Sundanese languages using both LLMs and human annotators. |
| Outcome: | The proposed model generates 4.5K questions per language, compared with 4.5k for Indonesian and 4.5km for Sundanese. |
PerCul: A Story-Driven Cultural Evaluation of LLMs in Persian (2025.naacl-long)
Copied to clipboard
Erfan Moosavi Monazzah, Vahid Rahimzadeh, Yadollah Yaghoobzadeh, Azadeh Shakery, Mohammad Taher Pilehvar
| Challenge: | Large language models predominantly reflect Western cultures due to the dominance of English-centric training data. |
| Approach: | They propose a dataset to assess the sensitivity of LLMs to Persian culture. |
| Outcome: | The proposed model shows a 11.3% gap between best closed-source model and layperson baseline while the gap increases to 21.3% by using the best open-weight model. |
GeoMLAMA: Geo-Diverse Commonsense Probing on Multilingual Pre-Trained Language Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent work shows that Pre-trained Language Models store relational knowledge and utilize it for performing downstream tasks. |
| Approach: | They propose a benchmark dataset for probing the diversity of relational knowledge in multilingual PLMs. |
| Outcome: | The proposed dataset contains 3125 prompts in English, Chinese, Hindi, Persian, and Swahili . larger multilingual PLMs variants do not store geo-diverse concepts better than its smaller variant . |
Broaden the Vision: Geo-Diverse Visual Commonsense Reasoning (2021.emnlp-main)
Copied to clipboard
| Challenge: | Generally, commonsense knowledge is correlated with culture and geographic locations and is only shared locally. |
| Approach: | They construct a Geo-Diverse Visual Commonsense Reasoning dataset to test vision-and-language models’ ability to understand cultural and geo-location-specific commonsense. |
| Outcome: | The proposed models perform better in non-Western regions including East Asia, South Asia, and Africa than in the Western regions. |
HellaSwag-Pro: A Large-Scale Bilingual Benchmark for Evaluating the Robustness of LLMs in Commonsense Reasoning (2025.findings-acl)
Copied to clipboard
Xiaoyuan Li, Moxin Li, Rui Men, Yichang Zhang, Keqin Bao, Wenjie Wang, Fuli Feng, Dayiheng Liu, Junyang Lin
| Challenge: | Existing studies show that large language models are robust in commonsense reasoning . however, some variations in questions can lead to incorrect responses . |
| Approach: | They propose a large-scale bilingual benchmark consisting of 11,200 cases . they conduct extensive experiments on 41 representative LLMs . |
| Outcome: | The proposed benchmark systematically evaluates the robustness of large language models in commonsense reasoning. |
On the Consistency of Commonsense in Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing evaluations of commonsense for large language models focus on downstream knowledge tasks, failing to probe whether LLMs truly understand and utilize knowledge or merely memorize it. |
| Approach: | They propose to automatically construct a large benchmark named CoCo which measures LLMs’ knowledge memorization, comprehension, and application and examines the consistency between these tasks. |
| Outcome: | The proposed benchmark systematically assesses LLMs’ knowledge memorization, comprehension, and application and examines the consistency between these tasks. |