Papers by Eleonora Presani
“I’m sorry to hear that”: Finding New Biases in Language Models with a Holistic Descriptor Dataset (2022.emnlp-main)
Copied to clipboard
| Challenge: | Language models are increasingly important to measure all possible demographic markers of identity . many datasets for measuring bias are limited in their coverage of demographic axes . |
| Approach: | They propose a bias measurement dataset that includes nearly 600 descriptor terms across 13 demographic axes. |
| Outcome: | The proposed dataset explores, detects, and reduces biases in language models. |
ROBBIE: Robust Bias Evaluation of Large Generative Language Models (2023.emnlp-main)
Copied to clipboard
David Esiobu, Xiaoqing Tan, Saghar Hosseini, Megan Ung, Yuchen Zhang, Jude Fernandes, Jane Dwivedi-Yu, Eleonora Presani, Adina Williams, Eric Smith
| Challenge: | generative large language models (LLMs) are becoming more performant and prevalent . we need tools to measure and improve their fairness, authors say . |
| Approach: | They propose to compare 6 different prompt-based bias and toxicity metrics across 12 demographic axes and 5 families of generative large language models. |
| Outcome: | The proposed model can be tested on more datasets to better characterize and mitigate biases . the study compared 6 prompt-based bias and toxicity metrics across 12 demographic axes and 5 families of generative large language models. |