Papers with IndiBias
BanStereoSet: A Dataset to Measure Stereotypical Social Biases in LLMs for Bangla (2025.findings-acl)
Copied to clipboard
| Challenge: | ***BanStereoSet*** is a dataset designed to evaluate stereotypical social biases in multilingual LLMs for the Bangla language. |
| Approach: | They propose to localize the content from StereoSet, IndiBias, and kamruzzaman-etal's datasets to capture biases prevalent within the Bangla language. |
| Outcome: | The proposed dataset consists of 1,194 sentences spanning 9 categories of bias: race, profession, gender, ageism, beauty, beauty in profession, region, caste, and religion. |
IndiBias: A Benchmark Dataset to Measure Social Biases in Language Models for Indian Context (2024.naacl-long)
Copied to clipboard
Nihar Sahoo, Pranamya Kulkarni, Arif Ahmad, Tanu Goyal, Narjis Asad, Aparna Garimella, Pushpak Bhattacharyya
| Challenge: | Existing benchmark datasets focus on English language and the Western context, leaving a void for a reliable dataset that encapsulates India’s unique socio-cultural nuances. |
| Approach: | They propose to use CrowS-Pairs to create a benchmark dataset that captures and evaluates social biases in Large Language Models (LLMs). |
| Outcome: | The proposed dataset is available in English and Hindi and leverages LLMs ChatGPT and InstructGPT to augment the existing dataset with diverse societal biases and stereotypes prevalent in India. |