Papers by Elizabeth Smith
Biasly: An Expert-Annotated Dataset for Subtle Misogyny Detection and Mitigation (2024.findings-acl)
Copied to clipboard
Brooklyn Sheppard, Anna Richter, Allison Cohen, Elizabeth Smith, Tamara Kneese, Carolyne Pelletier, Ioana Baldini, Yue Dong
| Challenge: | the Biasly dataset captures misogyny in movies in ways unique within the literature. |
| Approach: | The Biasly dataset captures misogyny in North American film by combining annotations of movie subtitles with common NLP algorithms. |
| Outcome: | The Biasly dataset captures misogyny expressions in North American film . it contains annotations of movie subtitles and text generation for rewrites . |
Sentence Mover’s Similarity: Automatic Evaluation for Multi-Sentence Texts (P19-1)
Copied to clipboard
| Challenge: | Existing automatic metrics for evaluating text are expensive and time-consuming. |
| Approach: | They propose automatic metrics that evaluate text in a continuous space using word and sentence embeddings. |
| Outcome: | The proposed method outperforms ROUGE on machine-generated summaries and human-authored essays on human-generated texts. |
All That’s ‘Human’ Is Not Gold: Evaluating Human Evaluation of Generated Text (2021.acl-long)
Copied to clipboard
| Challenge: | evaluators distinguish between human- and machine-authored text in three domains without training . evals' accuracy improved up to 55%, but it did not significantly improve across the three domain. |
| Approach: | They examine the role untrained human evaluations play in NLG evaluation and propose ways to improve their evaluations. |
| Outcome: | The evaluators distinguished between human- and machine-authored text at random chance level without training, but their accuracy did not improve across the three domains. |
Neural Text Generation in Stories Using Entity Representations as Context (N18-1)
Copied to clipboard
| Challenge: | Existing models of text generation that explicitly represent entities are based on the use of words and entities. |
| Approach: | They propose a neural model that explicitly represents entities mentioned in the text . they use vectors that are updated as the text proceeds to improve automatic evaluations . |
| Outcome: | The proposed model improves mention generation, sentence selection, and sentence generation. |
Choose Your Own Adventure: Paired Suggestions in Collaborative Writing for Evaluating Story Generation Models (2021.naacl-main)
Copied to clipboard
| Challenge: | Comparative evaluations have been shown to produce more reliable and consistent results than Likert scale ratings. |
| Approach: | They propose a collaborative writing setup where two models generate suggestions to people as they write a short story and then ask them to choose which model's suggestions they prefer. |
| Outcome: | The proposed model performs better in cases where the differences in generation methods are small (nucleus vs. top-k sampling) and large (GPT2 v. Fusion models). |
Sounding Board: A User-Centric and Content-Driven Social Chatbot (N18-5)
Copied to clipboard
Hao Fang, Hao Cheng, Maarten Sap, Elizabeth Clark, Ari Holtzman, Yejin Choi, Noah A. Smith, Mari Ostendorf
| Challenge: | Sounding Board is a social chatbot that can hold a coherent conversation with humans . the system is user-centric in that users can control the topic of conversation, while the system adapts to the user's needs. |
| Approach: | They present Sounding Board, a social chatbot that won the 2017 Amazon Alexa Prize. |
| Outcome: | The system is user-centric in that users can control the topic of conversation, while the system adapts to the user's needs. |