Papers by Patanamon Thongtanunam
CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | State-of-the-art large language models (LLMs) have demonstrated impressive code generation capabilities but struggle with real-world software engineering tasks such as revising source code to address code reviews. |
| Approach: | They propose a benchmark to evaluate large language models' ability to bridge both technical and conversational contexts by decomposing the generation task of code refinement into three essential reasoning steps. |
| Outcome: | The proposed benchmark exposes specific model weaknesses in code review comprehension disentangled from their generative automated code refinement results. |