Papers by Virginia Felkner
WinoQueer: A Community-in-the-Loop Benchmark for Anti-LGBTQ+ Bias in Large Language Models (2023.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks for measuring anti-LGBTQ+ bias are poorly defined and insufficiently grounded in real-world harms. |
| Approach: | They propose a bias benchmark that is community-sourced and generates a community survey. |
| Outcome: | The proposed method is community-sourced and improves on WinoQueer-v0. |
GPT is Not an Annotator: The Necessity of Human Annotation in Fairness Benchmark Construction (2024.acl-long)
Copied to clipboard
| Challenge: | Current benchmarks for social biases have limitations in scope, grounding, quality and human effort required. |
| Approach: | They propose to use a language model to help with the development of bias benchmarks . they extend previous work to a new community and set of biases: the Jewish community and antisemitism . |
| Outcome: | The proposed LLM does not perform well on the Jewish community and antisemitism task. |