Papers by Leroy Wang
APPLS: Evaluating Evaluation Metrics for Plain Language Summarization (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing evaluation metrics for plain language summarization (PLS) lack a dedicated assessment metric and the suitability of text generation evaluation metrics is unclear due to unique transformations. |
| Approach: | They propose a granular meta-evaluation testbed to evaluate PLS metrics . they identify four PLS criteria and define perturbations that sensitive metrics should be able to detect . |
| Outcome: | The proposed testbed assesses performance of 14 existing metrics including scores, features, and prompt-based evaluations. |
Whose Language Counts as High Quality? Measuring Language Ideologies in Text Data Selection (2022.emnlp-main)
Copied to clipboard
Suchin Gururangan, Dallas Card, Sarah Dreier, Emily Gade, Leroy Wang, Zeyu Wang, Luke Zettlemoyer, Noah A. Smith
| Challenge: | Language models rely on massive web crawls for diverse text data, but are rife with undesirable content. |
| Approach: | They analyze newspaper articles written by students from across the country to determine whose language is preferred by a quality filter. |
| Outcome: | The results show that newspapers from wealthier, educated, and urban zones are more likely to be classified as high quality. |