Investigating African-American Vernacular English in Transformer-Based Text Generation (2020.emnlp-main)
Copied to clipboard
Sophie Groenwold, Lily Ou, Aesha Parekh, Samhita Honnavalli, Sharon Levy, Diba Mirza, William Yang Wang
| Challenge: | Recent work in Natural Language Generation (NLG) uses a Transformer-based language model to generate high-quality, coherent text when prompted by arbitrary input. |
| Approach: | They evaluate the performance of a Transformer-based model that generates high-quality, coherent text when prompted by arbitrary input. |
| Outcome: | The proposed model improves on AAVE and SAE text with pretrained sentiment classifiers. |
Similar Papers
Evaluation of African American Language Bias in Natural Language Generation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have shown that large language generation models disadvantaging African American Language (AAL) can be biased for certain language varieties, but there is little research on the impact of these biases on other languages. |
| Approach: | They evaluate how well LLMs understand African American Language (AAL) in comparison to white Mainstream English (WME) using a dataset of AAL texts from a variety of regions and contexts, they find dialectal bias in six pre-trained LLM. |
| Outcome: | The proposed models understand African American language in comparison to white mainstream English (WME) the proposed models have performance gaps on two tasks that are not matched by the model. |
Twitter Universal Dependency Parsing for African-American and Mainstream American English (P18-1)
Copied to clipboard
| Challenge: | We analyze the performance disparities between AAE and Mainstream American English (MAE) because of Twitter-specific conventions and dialectal language. |
| Approach: | They develop a dataset of 500 tweets, 250 of which are in AAE, within the Universal Dependencies 2.0 framework and annotate it. |
| Outcome: | The proposed model improves performance for AAE tweets with no or very little in-domain labeled data and assesses its lexical and syntactic features. |
VALUE: Understanding Dialect Disparity in NLU (2022.acl-long)
Copied to clipboard
| Challenge: | English Natural Language Understanding systems outperform humans on benchmarks like GLUE and SuperGLUE, but they only use textbook Standard American English (SAE) . fewer studies have considered the effects of dialectal differences on performance . |
| Approach: | They propose a benchmark to evaluate the performance of English natural language understanding systems using a set of lexical and morphosyntactic transformation rules. |
| Outcome: | The proposed model outperforms humans on GLUE and SuperGLUE, but only on standard American English . the proposed model recruits fluent speakers of African American vernacular english to validate each feature transformation . |
Evaluating and Mitigating Inherent Linguistic Bias of African American English through Inference (2022.coling-1)
Copied to clipboard
| Challenge: | Recent studies show that NLP models trained on standard English produce biased outcomes against underrepresented English varieties. |
| Approach: | They propose a morphosyntactically-informed rule-based translation method that uses a greedy algorithm to debiase NLP models. |
| Outcome: | The proposed framework outperforms large language models while maintaining or improving the prediction performance. |
Leveraging Syntactic Dependencies in Disambiguation: The Case of African American English (2024.lrec-main)
Copied to clipboard
| Challenge: | African American English (AAE) is a low-resource language facing the challenge of inadequate annotated data for training natural language processing models. |
| Approach: | They propose a syntactically informed classifier for automatic disambiguation of AAE's habitual be. |
| Outcome: | The proposed classifier improves automatic disambiguation of habitual and non-habitual meanings of "be" integrating syntactic information improves disambiguations of habituality by 65 F1 points over baseline models and as much as 74 points. |
Analysis of LLM as a grammatical feature tagger for African American English (2025.findings-naacl)
Copied to clipboard
| Challenge: | African American English (AAE) presents unique challenges in natural language processing (NLP). |
| Approach: | They evaluate the ability of different NLP systems to recognize distinctive AAE grammatical features by using sentence-level binary classification tasks using both zero-shot and fewshot strategies. |
| Outcome: | The evaluation involved sentence-level binary classification tasks, using both zero-shot and few-shot strategies. |
Understanding the Impacts of Language Technologies’ Performance Disparities on African American Language Speakers (2024.findings-acl)
Copied to clipboard
| Challenge: | Previous work has examined performance disparities between AAL speakers and White Mainstream English speakers . but, this work has not sought to understand the impacts of these disparities on AAL speaker. |
| Approach: | They examine the experiences of African American Language (AAL) speakers when using language technologies. |
| Outcome: | The authors interview 19 AAL speakers to understand performance disparities . they find that speakers often undertake invisible labor to successfully use language technologies . |
Data Caricatures: On the Representation of African American Language in Pretraining Corpora (2025.acl-long)
Copied to clipboard
Nicholas Deas, Blake Vente, Amith Ananthram, Jessica A Grieser, Desmond U. Patton, Shana Kleiner, James R. Shepard Iii, Kathleen McKeown
| Challenge: | Recent work in linguistics and NLP has investigated the quantity and quality of AAL representation in pretraining corpora. |
| Approach: | They examine the quantity and quality of African American Language (AAL) representation in pretraining corpora. |
| Outcome: | The results show that AAL is underrepresented in all evaluated corpora compared to US demographics . they also show that most automated filters are more likely to conserve white Mainstream English (WME) texts over AAL . |
My LLM might Mimic AAE - But When Should It? (2025.naacl-long)
Copied to clipboard
| Challenge: | a study examines the representation of African American English in large language models . a survey of black americans and annotation of LLM outputs shows that Black Americans prefer to use AAE in formal settings . |
| Approach: | They examine Black Americans' perceptions of how effective AI tools are at producing authentic African American English in large language models. |
| Outcome: | The results show that Black Americans prefer to use LLMs in formal settings over informal ones . the results show they prefer to produce AAE in less formal settings . |
Synthetic Data for English Lexical Normalization: How Close Can We Get to Manually Annotated Data? (2020.lrec-1)
Copied to clipboard
| Challenge: | Social media data is a valuable data resource for natural language processing tasks. |
| Approach: | They propose to adapt input text to a more standard form, a task also referred to as normalization. |
| Outcome: | The proposed system scores 94.29 accuracy on the test data compared to 95.22 when trained on human-annotated data. |