VDebugger: Harnessing Execution Feedback for Debugging Visual Programs (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Visual programs are executable code generated by large language models to address visual reasoning problems. |
| Approach: | They propose a critic-refiner framework that localizes and debugs visual programs by tracking execution step by step. |
| Outcome: | The proposed framework detects and corrects program errors leveraging detailed execution feedback, improving interpretability and accuracy. |
Similar Papers
Debug like a Human: A Large Language Model Debugger via Verifying Runtime Execution Step by Step (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are leading progress in code generation, but they are underutilized in the literature. |
| Approach: | They propose a debugging framework that allows LLMs to refine their generated programs with the runtime execution information. |
| Outcome: | The proposed framework improves the baseline performance by 9.8% across the HumanEval, MBPP, and TransCoder benchmarks. |
Think Like You Execute: Verifiable Chain of Thought from Program Traces (2026.acl-industry)
Copied to clipboard
Shailja Thakur, Vaibhav Saxena, Rohan Kulkarni, Shivdeep Singh, Parameswaran Selvam, Hiroshi Kanayama, Hima Patel
| Challenge: | Current synthetic Chain-of-Thought (CoT) training data often consists of plausible-sounding explanations generated by teacher models, not verifiable accounts of actual program behavior. |
| Approach: | They propose to ground CoT generation directly in program execution traces to improve reasoning capabilities. |
| Outcome: | The proposed pipeline improves performance on live code benchmarks and on cruxEval-output and cruxeval-input. |
Extracting the Essence and Discarding the Dross: Enhancing Code Generation with Contrastive Execution Feedback (2025.coling-main)
Copied to clipboard
| Challenge: | erroneous code generation methods amalgamate feedback and correct code as target sentences . a new approach to code generation with feedback is needed to improve model performance . |
| Approach: | They propose a learning-based code generation model with execution feedback that integrates feedback and correct code as target sentences. |
| Outcome: | a new model with execution feedback shows improvements in generating accurate code and understanding error correction. |
VisualCoder: Guiding Large Language Models in Code Execution with Fine-grained Multimodal Chain-of-Thought Reasoning (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing approaches to enhance large language models' ability to predict program behavior struggle with dynamic reasoning tasks. |
| Approach: | They propose a visual control flow graph that integrates CoT reasoning with a control flow . they aim to improve performance in program behavior prediction, error detection and output generation . |
| Outcome: | The proposed approach improves program behavior prediction, error detection, and output generation. |
Revisit Self-Debugging with Self-Generated Tests for Code Generation (2025.acl-long)
Copied to clipboard
Xiancai Chen, Zhengwei Tao, Kechi Zhang, Changzhi Zhou, Xinyu Zhang, Wanli Gu, Yuanpeng He, Mengdi Zhang, Xunliang Cai, Haiyan Zhao, Zhi Jin
| Challenge: | Large language models (LLMs) have made significant advances in code generation, but they still face challenges when tackling complex programming tasks beyond their basic capabilities. |
| Approach: | They propose to integrate self-generated tests into the code generation process . they propose to use post-execution and in-exection self-debugging to mitigate test bias . |
| Outcome: | The proposed method improves the performance of large language models in code generation tasks by leveraging execution feedback from tests. |
The Path Not Taken: Duality in Reasoning about Program Execution (2026.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks focus on predicting program properties tied to specific inputs, resulting in a narrow view of dynamic code reasoning and data contamination. |
| Approach: | They instantiate dual-path reasoning in a benchmark and evaluate 13 LLMs. |
| Outcome: | The proposed model provides a robust and discriminative proxy for dynamic code understanding. |
UniDebugger: Hierarchical Multi-Agent Framework for Unified Software Debugging (2025.emnlp-main)
Copied to clipboard
Cheryl Lee, Chunqiu Steven Xia, Longji Yang, Jen-tse Huang, Zhouruixing Zhu, Lingming Zhang, Michael R. Lyu
| Challenge: | Existing LLMs focus on isolated steps and struggle with complex bugs. |
| Approach: | They propose a framework for unified debugging through multi-agent synergy . it mimics the entire cognitive processes of developers with each agent specialized as a particular component of this process . |
| Outcome: | The proposed framework outperforms state-of-the-art methods on repo-level benchmarks. |
Detect-Localize-Repair: A Unified Framework for Learning to Debug with CodeT5 (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Automated software debugging is crucial for improving productivity of software developers . many neural-based techniques focus only on one or the other, ignoring mutual benefits . |
| Approach: | They propose a framework to adapt a pretrained programming language model to automate debugging . they propose three objectives: bug detection, bug localization, program repair . |
| Outcome: | The proposed framework outperforms baselines from both NLP and software engineering domains on two new datasets. |
DebugBench: Evaluating Debugging Capability of Large Language Models (2024.findings-acl)
Copied to clipboard
Runchu Tian, Yining Ye, Yujia Qin, Xin Cong, Yankai Lin, Yinxu Pan, Yesai Wu, Hui Haotian, Liu Weichuan, Zhiyuan Liu, Maosong Sun
| Challenge: | Large language models (LLMs) have demonstrated exceptional coding capabilities, but their debugging capabilities remain relatively unexplored. |
| Approach: | They propose a debugging benchmark consisting of 4,253 LLMs with four major bug categories and 18 minor types in C++, Java, and Python. |
| Outcome: | The proposed benchmark covers four major bug categories and 18 minor types in C++, Java, and Python. |
Program Synthesis Benchmark for Visual Programming in XLogoOnline Environment (2025.acl-long)
Copied to clipboard
| Challenge: | Large language and multimodal models have shown remarkable success on various benchmarks focused on specific skills such as general-purpose programming, math word problem-solving, and visual question answering. |
| Approach: | They propose a program synthesis benchmark based on real-world programming tasks . they propose 'fine-tuning pipeline' to boost performance of large language models . |
| Outcome: | The proposed model outperforms existing models on tasks that require a combination of skills on visual programming and programming. |