Papers by Shana Kleiner
Data Caricatures: On the Representation of African American Language in Pretraining Corpora (2025.acl-long)
Copied to clipboard
Nicholas Deas, Blake Vente, Amith Ananthram, Jessica A Grieser, Desmond U. Patton, Shana Kleiner, James R. Shepard Iii, Kathleen McKeown
| Challenge: | Recent work in linguistics and NLP has investigated the quantity and quality of AAL representation in pretraining corpora. |
| Approach: | They examine the quantity and quality of African American Language (AAL) representation in pretraining corpora. |
| Outcome: | The results show that AAL is underrepresented in all evaluated corpora compared to US demographics . they also show that most automated filters are more likely to conserve white Mainstream English (WME) texts over AAL . |
Evaluation of African American Language Bias in Natural Language Generation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have shown that large language generation models disadvantaging African American Language (AAL) can be biased for certain language varieties, but there is little research on the impact of these biases on other languages. |
| Approach: | They evaluate how well LLMs understand African American Language (AAL) in comparison to white Mainstream English (WME) using a dataset of AAL texts from a variety of regions and contexts, they find dialectal bias in six pre-trained LLM. |
| Outcome: | The proposed models understand African American language in comparison to white mainstream English (WME) the proposed models have performance gaps on two tasks that are not matched by the model. |