Papers by Johannes Schäfer
Which Demographics do LLMs Default to During Annotation? (2025.acl-long)
Copied to clipboard
Johannes Schäfer, Aidan Combs, Christopher Bagdon, Jiahui Li, Nadine Probol, Lynn Greschner, Sean Papay, Yarik Menchaca Resendiz, Aswathy Velutharambath, Amelie Wuehrl, Sabine Weber, Roman Klinger
| Challenge: | Demographics and cultural background of annotators influence the labels they assign in text annotation. |
| Approach: | They examine the attributes of human annotators LLMs inherently mimic and compare them to demographic-conditioned prompts and placebo-conditioned ones. |
| Outcome: | The proposed model incorporates demographics and cultural background into the output of the large language models (LLMs) to evaluate which attributes of human annotators LLMs inherently mimic. |