Papers by Thanh Le-Cong
Can LLMs Reason About Program Semantics? A Comprehensive Evaluation of LLMs on Formal Specification Inference (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly being used to automate programming tasks. |
| Approach: | They propose a benchmark to evaluate LLMs' reasoning abilities on program semantics. |
| Outcome: | The proposed benchmark shows that LLMs perform well with simple control flows but struggle with more complex structures, especially loops, even with advanced prompting. |