Papers by Joost Noppen
A Study of LLMs’ Preferences for Libraries and Programming Languages (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing evaluations focus on functional correctness or syntactic validity, overlooking how LLMs make critical design choices such as which library or programming language to use. |
| Approach: | They conduct the first systematic study of LLMs’ preferences for libraries and programming languages when generating code, covering eight different LLM. |
| Outcome: | The proposed benchmarks show that LLMs prioritize familiarity and popularity over suitability and task-specific optimality. |