On The Origin of Cultural Biases in Language Models: From Pre-training Data to Linguistic Phenomena (2025.naacl-long)
Copied to clipboard
| Challenge: | Language Models (LMs) have been shown to exhibit a strong preference towards entities associated with Western culture when operating in non-Western languages. |
| Approach: | They propose a parallel Arabic-English benchmark of 58,086 entities associated with Arab and Western cultures and 367 masked natural contexts for entities. |
| Outcome: | The proposed model shows that LMs struggle in Arabic with entities that appear at high frequencies in pre-training, where entities can hold multiple word senses. |
Similar Papers
From n-gram to Attention: How Model Architectures Learn and Propagate Bias in Language Modeling (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Current research on bias in language models focuses on data quality, not temporal influences of data. |
| Approach: | They propose a methodology to interpret the interaction between training data and model architecture in bias propagation during language modeling. |
| Outcome: | The proposed method analyzes the interaction between training data and model architecture in bias propagation during language modeling. |
A Predictive Factor Analysis of Social Biases and Task-Performance in Pretrained Masked Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Various types of social biases have been reported with pretrained Masked Language Models (MLMs) in prior work. |
| Approach: | They conduct a comprehensive study on 39 pretrained MLMs to examine their model factors and their social biases. |
| Outcome: | The proposed model factors influence social biases learned by an MLM and their downstream task performance. |
“You are grounded!”: Latent Name Artifacts in Pre-trained Language Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained language models perpetuate biases originating in their training corpus to downstream models. |
| Approach: | They focus on the representations of given names in pre-trained language models and show that name perturbation can have an effect on downstream tasks. |
| Outcome: | The proposed model can be used to model the representation of given names in pre-trained language models on reading comprehension probes where name perturbation changes the model answers. |
Global Voices, Local Biases: Socio-Cultural Prejudices across Languages (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on human biases are heavily skewed towards Western and European languages . despite growing interest in language models, there are several shortcomings in the literature . |
| Approach: | They scale the Word Embedding Association Test to 24 languages and add culturally relevant information for each language. |
| Outcome: | The proposed language models can reflect and often amplify the effects of bias across linguistic, cultural, and societal borders. |
From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models (2023.acl-long)
Copied to clipboard
| Challenge: | Hundreds of studies have highlighted ethical issues in NLP models . |
| Approach: | They propose to measure media biases in LMs trained on diverse data sources . they focus on hate speech and misinformation detection . |
| Outcome: | The proposed methods quantify the fairness of downstream NLP models trained on politically biased LMs. |
Revisiting Pre-trained Language Models and their Evaluation for Arabic Natural Language Processing (2022.emnlp-main)
Copied to clipboard
Abbas Ghaddar, Yimeng Wu, Sunyam Bagga, Ahmad Rashid, Khalil Bibi, Mehdi Rezagholizadeh, Chao Xing, Yasheng Wang, Xinyu Duan, Zhefeng Wang, Baoxing Huai, Xin Jiang, Qun Liu, Phillippe Langlais
| Challenge: | Existing pre-trained language models are not well-explored and are not reproducible in the literature. |
| Approach: | They propose to improve existing Arabic language pre-trained language models using a more methodical approach. |
| Outcome: | The proposed models outperform existing models on ALUE, a leaderboard-powered benchmark for Arabic NLU and NLG tasks. |
Social Bias in Multilingual Language Models: A Survey (2025.emnlp-main)
Copied to clipboard
| Challenge: | Pretrained multilingual models exhibit the same social bias as models processing English texts. |
| Approach: | They examine the literature on bias evaluation and mitigation approaches in multilingual and non-English contexts and identify gaps in the field. |
| Outcome: | The proposed models perform well on multilingual language understanding benchmarks and are consistent with the current literature. |
Comparing Biases and the Impact of Multilingual Training across Multiple Languages (2023.emnlp-main)
Copied to clipboard
Sharon Levy, Neha John, Ling Liu, Yogarshi Vyas, Jie Ma, Yoshinari Fujinuma, Miguel Ballesteros, Vittorio Castelli, Dan Roth
| Challenge: | Currently, studies on bias and fairness in natural language processing focus on a single language and/or across few attributes (e.g. gender, race). However, biases can manifest differently across languages for individual attributes. |
| Approach: | They adapt existing sentiment bias templates in English to Italian, Chinese, Hebrew, and Spanish for race, religion, nationality, and gender. |
| Outcome: | The proposed model favors groups that are dominant in each language's culture, indicating bias amplification, after multilingual finetuning. |
Addressing Healthcare-related Racial and LGBTQ+ Biases in Pretrained Language Models (2024.findings-naacl)
Copied to clipboard
| Challenge: | Pretrained language models (PLMs) propagate social stigmas and stereotypes, a critical concern given their widespread use. |
| Approach: | They adapt two intrinsic bias benchmarks to quantify racial and LGBTQ+ biases in prevalent PLMs and empirically evaluate the effectiveness of various debiasing methods in mitigating these biase. |
| Outcome: | The proposed methods reduce biases without compromising performance in downstream tasks. |
Palm: A Culturally Inclusive and Linguistically Diverse Dataset for Arabic LLMs (2025.acl-long)
Copied to clipboard
Fakhraddin Alwajih, Abdellah El Mekki, Samar Mohamed Magdy, AbdelRahim A. Elmadany, Omer Nacar, El Moatez Billah Nagoudi, Reem Abdel-Salam, Hanin Atwany, Youssef Nafea, Abdulfattah Mohammed Yahya, Rahaf Alhamouri, Hamzah A. Alsayadi, Hiba Zayed, Sara Shatnawi, Serry Sibaee, Yasir Ech-chammakhy, Walid Al-Dhabyani, Marwa Mohamed Ali, Imen Jarraya, Ahmed Oumar El-Shangiti, Aisha Alraeesi, Mohammed Anwar AL-Ghrawi, Abdulrahman S. Al-Batati, Elgizouli Mohamed, Noha Taha Elgindi, Muhammed Saeed, Houdaifa Atou, Issam Ait Yahia, Abdelhak Bouayad, Mohammed Machrouh, Amal Makouar, Dania Alkawi, Mukhtar Mohamed, Safaa Taher Abdelfadil, Amine Ziad Ounnoughene, Anfel Rouabhia, Rwaa Assi, Ahmed Sorkatti, Mohamedou Cheikh Tourad, Anis Koubaa, Ismail Berrada, Mustafa Jarrar, Shady Shehata, Muhammad Abdul-Mageed
| Challenge: | a year-long community-driven project covering all 22 Arab countries evaluates the cultural and dialectal capabilities of large language models. |
| Approach: | They propose a project to evaluate the cultural and dialectal capabilities of large language models. |
| Outcome: | The project evaluates the cultural and dialectal capabilities of several frontier LLMs. |