Papers by Li Lucy
“One-Size-Fits-All”? Examining Expectations around What Constitute “Fair” or “Good” NLG System Behaviors (2024.naacl-long)
Copied to clipboard
| Challenge: | Natural language generation models are used for many downstream applications involving interpersonal communication, such as text completion, "smart" reply suggestions, and chatbot assistants. |
| Approach: | They conduct five case studies that perturb identity-related language features in NLG inputs to examine their assumptions about fairness. |
| Outcome: | The findings highlight open challenges around what constitutes “fair” or “good” NLG system behaviors. |
Tell, Don’t Show: Leveraging Language Models’ Abstractive Retellings to Model Literary Themes (2025.findings-acl)
Copied to clipboard
| Challenge: | Literature challenges traditional bag-of-words approaches for topic modeling because narrative language focuses on immersive sensory details instead of abstractive description or exposition. |
| Approach: | They propose a topic modeling approach that prompts generative language models to *tell* what passages *show*, thereby translating narratives’ surface forms into higher-level concepts and themes. |
| Outcome: | The proposed model can translate narratives’ surface forms into higher-level concepts and themes than by running LDA alone or directly asking LMs to list topics. |
MathFish: Evaluating Language Model Math Reasoning via Grounding in Educational Curricula (2024.findings-emnlp)
Copied to clipboard
| Challenge: | pedagogical experts spend months reviewing published math problems to ensure that they align with critical skills or concepts. |
| Approach: | They propose a novel approach for evaluating language models' mathematical abilities by combining a dataset of 385 fine-grained descriptions of K-12 math skills and concepts with 9.9K math problems labeled with these standards. |
| Outcome: | The proposed model can discern skills and concepts enabled by math content, and it can be used to assess language models' mathematical abilities. |
Characterizing English Variation across Social Media Communities with BERT (2021.tacl-1)
Copied to clipboard
| Challenge: | Existing studies characterizing language variation across Internet social groups have focused on the types of words used by these groups. |
| Approach: | They extend this study by employing BERT to characterize variation in the senses of words as well, analyzing two months of English comments in 474 Reddit communities. |
| Outcome: | The proposed study analyzes two months of English comments in 474 Reddit communities and ties language variation with community behavior. |
DrawEduMath: Evaluating Vision Language Models with Expert-Annotated Students’ Hand-Drawn Math Images (2025.naacl-long)
Copied to clipboard
| Challenge: | DrawEduMath examines the ability of vision language models to handle real-world math problems, such as those encountered in classrooms and tutoring sessions. |
| Approach: | They present DrawEduMath, an English-language dataset of 2,030 images of students’ handwritten responses to math problems. |
| Outcome: | The proposed model can be used to evaluate teachers' QA pairs and 44,362 synthetic QAs derived from teachers' descriptions. |
Words as Gatekeepers: Measuring Discipline-specific Terms and Meanings in Scholarly Publications (2023.findings-acl)
Copied to clipboard
| Challenge: | Specialized terminology, or specialized language, can facilitate efficient in-group communication within fields but hinder understanding for out-groups. |
| Approach: | They propose an interpretable approach to measure scholarly jargon from text using word sense induction to identify words that are widespread but overloaded with different meanings across fields. |
| Outcome: | The proposed method is validated by word sense induction and shows that it provides a complementary, yet unique view of jargon alongside word types. |
AboutMe: Using Self-Descriptions in Webpages to Document the Effects of English Pretraining Data Filters (2024.acl-long)
Copied to clipboard
| Challenge: | Large language models' (LLMs) abilities are drawn from their pretraining data. however, decisions around what data is retained or removed during this initial stage are under-scrutinized. |
| Approach: | They ground web text, a popular pretraining data source, to its social and geographic contexts. |
| Outcome: | The results show that some quality classifiers act like topical domain filters, and langID overlook English content from some regions of the world. |
Discovering Differences in the Representation of People using Contextualized Semantic Axes (2022.emnlp-main)
Copied to clipboard
| Challenge: | Past work has compared embeddings against “semantic axes” that represent two opposing concepts. |
| Approach: | They extend this paradigm to BERT embeddings and construct contextualized axes that mitigate pitfall where antonyms have neighboring representations. |
| Outcome: | The proposed axes can characterize differences among instances of the same word type on two people-centric datasets. |