Papers by Emma Pierson
Topics, Authors, and Institutions in Large Language Model Research: Trends from 17K arXiv Papers (2024.naacl-long)
Copied to clipboard
| Challenge: | Recent advances in language modeling have caused disruptive shifts throughout AI research, spurring discussion about how the field is changing and how it should change. |
| Approach: | They analyze a dataset of 16,979 LLM-related arXiv papers and examine industry and academic publishing trends. |
| Outcome: | The authors examine the impact of large language models on AI research in 2023 and 2022. |
Annotation alignment: Comparing LLM and human annotations of conversational safety (2024.emnlp-main)
Copied to clipboard
| Challenge: | We examine whether LLMs and humans agree when annotating the safety of user-chatbot conversations. |
| Approach: | They leverage a recent DICES dataset in which 350 conversations are each rated for safety by 112 annotators spanning 10 race-gender groups. |
| Outcome: | The LLMs annotators are compared to human annotator demographic groups and can predict when one group finds a conversation unsafe . |