Papers by Hannah Babe
StudentEval: A Benchmark of Student-Written Prompts for Large Language Models of Code (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing CodeLLM benchmarks rely on a single expert-written prompt per problem . a growing body of work shows their utility to professional programmers . |
| Approach: | They propose a natural-language-to-code benchmark of prompts written by non-experts . student prompts are written by 80 students who have only completed one introductory Python course . |
| Outcome: | The proposed model is better discriminator of student prompt descriptions than existing benchmarks. |