Papers by Rob Kwiatkowski
A Static Evaluation of Code Completion by Large Language Models (2023.acl-industry)
Copied to clipboard
Hantian Ding, Varun Kumar, Yuchen Tian, Zijian Wang, Rob Kwiatkowski, Xiaopeng Li, Murali Krishna Ramanathan, Baishakhi Ray, Parminder Bhatia, Sudipta Sengupta
| Challenge: | Large language models trained on code have shown great potential to increase productivity of software developers. |
| Approach: | They propose a static evaluation framework to quantify static errors in Python code completions by leveraging Abstract Syntax Trees. |
| Outcome: | The proposed framework is more efficient and applicable to code in the wild. |