Papers by Huaman Sun
Sociodemographic Prompting is Not Yet an Effective Approach for Simulating Subjective Judgments with LLMs (2025.naacl-short)
Copied to clipboard
| Challenge: | Large language models (LLMs) are widely used to simulate human responses, but their ability to account for demographic differences in subjective tasks remains uncertain. |
| Approach: | They evaluate large language models' ability to understand demographic differences in two subjective judgment tasks: politeness and offensiveness. |
| Outcome: | The proposed models perform better in politeness and offensiveness tasks, while sociodemographic prompting does not improve and worsens their ability to perceive language from sub-populations. |