Spurious Correlations and Beyond: Understanding and Mitigating Shortcut Learning in SDOH Extraction with Large Language Models (2025.acl-short)
Copied to clipboard
| Challenge: | Large language models (LLMs) rely on superficial cues leading to spurious predictions . recent work has highlighted how LLMs exploit spurious patterns rather than learning causal, generalizable features. |
| Approach: | They use a social history annotation corpus dataset to examine drug status extraction . they evaluate prompt engineering and chain-of-thought reasoning to reduce false positives . |
| Outcome: | The proposed model can predict drug use when alcohol or smoking is not present, while uncovering gender disparities in model performance. |
Similar Papers
SDOH-NLI: a Dataset for Inferring Social Determinants of Health from Clinical Notes (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Social and behavioral determinants of health (SDOH) play a significant role in shaping health outcomes, and extracting these determinant from clinical notes is a first step to help healthcare providers systematically identify opportunities to provide appropriate care and address disparities. |
| Approach: | They propose a dataset that extracts social and behavioral determinants from clinical notes and uses them to form a natural language inference task. |
| Outcome: | The proposed dataset is based on publicly available notes and is more challenging than standard NLI benchmarks. |
Extracting Social Determinants of Health from Pediatric Patient Notes Using Large Language Models: Novel Corpus and Methods (2024.lrec-main)
Copied to clipboard
Yujuan Fu, Giridhar Kaushik Ramachandran, Nicholas J. Dobbins, Namu Park, Michael Leu, Abby R. Rosenberg, Kevin Lybarger, Fei Xia, Özlem Uzuner, Meliha Yetisgen
| Challenge: | Social determinants of health (SDoH) are often studied in the electronic health record (EHR) however, there are difficulties in documenting SDoH in a tabular format due to the lack of a comprehensive SDoh tool. |
| Approach: | They propose to annotate social history sections from 1,260 clinical notes from pediatric patients within the University of Washington (UW) hospital system. |
| Outcome: | The proposed corpus captures ten distinct health determinants including living and economic stability, prior trauma, education access, substance use history, and mental health with an overall annotator agreement of 81.9 F1. |
Investigating Gender Stereotypes in Large Language Models via Social Determinants of Health (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing benchmarks evaluate biases related to individual social determinants of health (SDoH) but they overlook interactions between these factors and lack context-specific assessments. |
| Approach: | They investigated the relationship between gender and other SDoH in french patient records to determine whether LLMs rely on embedded stereotypes to make gendered decisions. |
| Outcome: | The proposed models can probe stereotypes and make gendered decisions based on the data. |
Identifying and Mitigating Spurious Correlations for Improving Robustness in NLP Models (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing work identifies task-specific shortcuts via human priors or error analyses, which requires extensive expertise and efforts. |
| Approach: | They propose to automatically identify spurious correlations in NLP models at scale by using existing interpretability methods to extract tokens that significantly affect model’s decision process. |
| Outcome: | The proposed method can identify spurious correlations in NLP models at scale and mitigate these leads to more robust models in multiple applications. |
Unexpected Phenomenon: LLMs’ Spurious Associations in Information Extraction (2024.findings-acl)
Copied to clipboard
Weiyan Zhang, Wanpeng Lu, Jiacheng Wang, Yating Wang, Lihan Chen, Haiyun Jiang, Jingping Liu, Tong Ruan
| Challenge: | Information extraction (IE) tasks require a limited number of example instructions to achieve effective performance. |
| Approach: | They propose two strategies to find spurious associations in large language models (LLMs) they use forward label extension and backward label validation to leverage extended labels to improve model performance. |
| Outcome: | The proposed methods improve performance on Chinese and English datasets and 9.55%, 11.42%, and 21.27% in F1 scores on SciERC, ACE05, and DuEE datasets. |
Reducing Spurious Correlations in Aspect-based Sentiment Analysis with Explanation from Large Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Aspect-based sentiment analysis models are susceptible to learning spurious correlations between words . a recent study shows that feature engineering is time-consuming and costly . |
| Approach: | They propose to use a template to prompt LLMs to generate an appropriate explanation for the sentiment polarity of each aspect to reduce spurious correlations. |
| Outcome: | The proposed methods improve ABSA models and their generalization ability. |
Addressing Healthcare-related Racial and LGBTQ+ Biases in Pretrained Language Models (2024.findings-naacl)
Copied to clipboard
| Challenge: | Pretrained language models (PLMs) propagate social stigmas and stereotypes, a critical concern given their widespread use. |
| Approach: | They adapt two intrinsic bias benchmarks to quantify racial and LGBTQ+ biases in prevalent PLMs and empirically evaluate the effectiveness of various debiasing methods in mitigating these biase. |
| Outcome: | The proposed methods reduce biases without compromising performance in downstream tasks. |
Exploring and Mitigating Shortcut Learning for Generative Large Language Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent large language models (LLMs) have incredible instruction-following capabilities while maintaining strong task completion ability. |
| Approach: | They propose a framework to encourage LLMs to Forget Spurious correlations and Learn from In-context information. |
| Outcome: | The proposed framework can mitigate shortcut learning by forging spurious correlations and learning from in-context information. |
Analyzing Biases to Spurious Correlations in Text Classification Tasks (2022.aacl-short)
Copied to clipboard
| Challenge: | Often these systems exceed human performance, but there is a caveat: standard benchmarks often assume that training and evaluation data are drawn independently and identically from the same underlying distribution. |
| Approach: | They propose to exploit spurious correlations in training data to exploit these correlations . they show that even when only ‘stop’ words are available, it is possible to predict the class significantly better than random. |
| Outcome: | The proposed model can predict class significantly better when only ‘stop’ words are available at the input stage, but can degrade the ability of the system to generalize well to out-of-domain data. |
Building Trust in Clinical LLMs: Bias Analysis and Dataset Transparency (2025.emnlp-main)
Copied to clipboard
Svetlana Maslenkova, Clement Christophe, Marco AF Pimentel, Tathagata Raha, Muhammad Umar Salman, Ahmed Al Mahrooqi, Avani Gupta, Shadab Khan, Ronnie Rajan, Praveenkumar Kanithi
| Challenge: | Current dataset curation and bias assessment practices lack transparency . current approaches lack a thorough understanding of how data characteristics influence model behavior . |
| Approach: | They propose a comprehensive bias evaluation framework that integrates general benchmarks with a healthcare-specific methodology to probe for biases in a sensitive healthcare context. |
| Outcome: | The proposed approach to bias evaluation leverages established benchmarks and a healthcare-specific methodology. |