Papers by Chahat Raj
VIGNETTE: Socially Grounded Bias Evaluation for Vision-Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing studies on VLM bias focus on portrait-style images and gender-occupation associations . existing studies ignore broader and more complex social stereotypes and their implied harm . |
| Approach: | They propose a large-scale VQA benchmark for evaluating bias in vision-language models . they use a question-answering framework that spans factuality, perception, stereotyping, and decision making . |
| Outcome: | The proposed framework examines bias in vision-language models using 30M+ images . findings reveal subtle, multifaceted, and surprising stereotypical patterns . |
Toward Inclusive Language Models: Sparsity-Driven Calibration for Systematic and Interpretable Mitigation of Social Biases in LLMs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a new method to mitigate stereotypical bias in large language models is needed . inherent biases from training on vast Internet datasets can amplify harmful stereotypes . |
| Approach: | They propose a method to identify stereotypical bias in decoder-only transformer models . they apply a localization mechanism that correlates internal activations with a new Context Influence score . |
| Outcome: | The proposed method reduces stereotypical biases on BBQ, StereoSet, and CrowS-Pairs while improving reasoning performance on MMLU by 10%. |
Talent or Luck? Evaluating Attribution Bias in Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on social biases in large language models focus on surface-level associations or isolated stereotypes. |
| Approach: | They propose a cognitively grounded bias evaluation framework to capture demographic biases across three contexts: single-actor, actor–actor and actor–observer. |
| Outcome: | The proposed framework captures comparative and perspective-driven biases overlooked in previous work. |
BiasDora: Exploring Hidden Biased Associations in Vision-Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies on social biases focus on a limited set of documented associations, such as gender-profession or race-crime. |
| Approach: | They propose to examine hidden, implicit bias associations across 9 bias dimensions by probing VLMs to uncover hidden, unexamined associations. |
| Outcome: | The proposed methods reveal that biases vary in negativity, toxicity, and extremity. |
What’s Not Said Still Hurts: A Description-Based Evaluation Framework for Measuring Social Bias in LLMs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing benchmarks evaluate bias by term-based mode, but they fail to capture hidden biases in nuanced settings. |
| Approach: | They propose a dataset to assess bias at the semantic level that bias concepts are hidden within naturalistic, subtly framed contexts in real-world scenarios. |
| Outcome: | The proposed dataset shows that models reduce bias in response at term level, but reinforce bias in nuanced settings. |
Global Voices, Local Biases: Socio-Cultural Prejudices across Languages (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on human biases are heavily skewed towards Western and European languages . despite growing interest in language models, there are several shortcomings in the literature . |
| Approach: | They scale the Word Embedding Association Test to 24 languages and add culturally relevant information for each language. |
| Outcome: | The proposed language models can reflect and often amplify the effects of bias across linguistic, cultural, and societal borders. |