Papers by Gilsinia Lopez
Stereotyping Norwegian Salmon: An Inventory of Pitfalls in Fairness Benchmark Datasets (2021.acl-long)
Copied to clipboard
| Challenge: | Several recent efforts have focused on benchmark datasets consisting of pairs of contrastive sentences, which are often accompanied by metrics that aggregate an NLP system’s behavior on these pairs into measurements of harms. |
| Approach: | They apply a measurement modeling lens to inventory pitfalls that threaten benchmarks' validity as measurement models for stereotyping. |
| Outcome: | The proposed benchmarks lack clarity and assumptions that affect how they conceptualize and operationalize stereotyping. |