DuET: Dual Execution for Test Output Prediction with Generated Code and Pseudocode (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have shown that test output prediction is difficult to achieve due to code errors. |
| Approach: | They propose a framework that grounds prediction on error-resilient pseudocode and simulates execution via LLM reasoning to overcome limitations of direct execution suffering from code errors. |
| Outcome: | The proposed framework improves Pass@1 on LiveCodeBench, BigCodeBech-Hard, DevEval and HumanEval(+) and improves on pass@1 by 13.6 pp. |
Similar Papers
PairCoder: Pair Programming-Inspired Two-Agent Collaboration for Code Generation (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing multi agent frameworks for large language models are brittle on code generation tasks. |
| Approach: | They propose a framework that brings pair programming to autonomous LLM collaboration. |
| Outcome: | Using PairCoder, large language models achieve better results on code generation tasks and reduce token usage by 40% to 70% on eight representative backbones. |
Revisit Self-Debugging with Self-Generated Tests for Code Generation (2025.acl-long)
Copied to clipboard
Xiancai Chen, Zhengwei Tao, Kechi Zhang, Changzhi Zhou, Xinyu Zhang, Wanli Gu, Yuanpeng He, Mengdi Zhang, Xunliang Cai, Haiyan Zhao, Zhi Jin
| Challenge: | Large language models (LLMs) have made significant advances in code generation, but they still face challenges when tackling complex programming tasks beyond their basic capabilities. |
| Approach: | They propose to integrate self-generated tests into the code generation process . they propose to use post-execution and in-exection self-debugging to mitigate test bias . |
| Outcome: | The proposed method improves the performance of large language models in code generation tasks by leveraging execution feedback from tests. |
S*: Test Time Scaling for Code Generation (2025.findings-emnlp)
Copied to clipboard
Dacheng Li, Shiyi Cao, Chengkun Cao, Xiuyu Li, Shangyin Tan, Kurt Keutzer, Jiarong Xing, Joseph E. Gonzalez, Ion Stoica
| Challenge: | S* is the first hybrid test-time scaling framework that significantly improves the coverage and selection accuracy of generated code. |
| Approach: | They propose a hybrid test-time scaling framework that augments parallel scaling with sequential scaling to further increase the performance. |
| Outcome: | The proposed framework outperforms existing scaling approaches in large-scale modeling and reasoning models. |
The Path Not Taken: Duality in Reasoning about Program Execution (2026.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks focus on predicting program properties tied to specific inputs, resulting in a narrow view of dynamic code reasoning and data contamination. |
| Approach: | They instantiate dual-path reasoning in a benchmark and evaluate 13 LLMs. |
| Outcome: | The proposed model provides a robust and discriminative proxy for dynamic code understanding. |
Self-Edit: Fault-Aware Code Editor for Code Generation (2023.acl-long)
Copied to clipboard
| Challenge: | Existing Large language models (LLMs) have low pass rates and accuracy on competitive programming tasks. |
| Approach: | They propose a generate-and-edit approach that uses execution results of generated code from LLMs to improve code quality on competitive programming tasks. |
| Outcome: | The proposed method improves pass@1 by 89% on APPS-dev, 31% on apps-test, and 48% on HumanEval over nine popular code generation LLMs with parameter sizes ranging from 110M to 175B. |
Beyond Code Pairs: Dialogue-Based Data Generation for LLM Code Translation (2026.acl-long)
Copied to clipboard
Le Chen, Nuo Xu, Winson Chen, Bin Lei, Pei-Hung Lin, Dunzhi Zhou, Rajeev Thakur, Caiwen Ding, Ali Jannesari, Chunhua Liao
| Challenge: | Large language models (LLMs) have shown remarkable capabilities in code translation, yet their performance deteriorates in low-resource programming domains such as Fortran and emerging frameworks like CUDA . |
| Approach: | They propose a dual-LLM Questioner–Solver pipeline that integrates external knowledge from compilers and runtime feedback to generate verified translations and multi-turn dialogues. |
| Outcome: | The proposed model outperforms proprietary models on key metrics like compilation success and accuracy. |
Harnessing Consistency for Robust Test-Time LLM Ensemble (2026.findings-eacl)
Copied to clipboard
Zhichen Zeng, Qi Yu, Xiao Lin, Ruizhong Qiu, Xuying Ning, Tianxin Wei, Yuchen Yan, Jingrui He, Hanghang Tong
| Challenge: | Existing efforts to improve LLM ensemble quality have focused on model consistency, but failures are often due to heterogeneous tokenization schemes and varying model expertise. |
| Approach: | They propose a plug-and-play technique that harnesses model consistency for robust LLM ensemble. |
| Outcome: | The proposed technique improves ensemble performance and robustness against erroneous signals. |
PLD+: Accelerating LLM Inference by Leveraging Language Model Artifacts (2025.findings-naacl)
Copied to clipboard
| Challenge: | speculative decoding is a novel decoding paradigm for large language models . however, its use is limited by its computational resources and fine-tuning requirements . |
| Approach: | They propose a tuning-free approach that accelerates inference of large language models . they use draft and verify principle to accelerate inference process . |
| Outcome: | The proposed approach outperforms tuning-free approaches on input-guided tasks and outperformed state-of-the-art EAGLE on four of the tasks. |
CoRE: A Fine-Grained Code Reasoning Benchmark Beyond Output Prediction (2026.findings-acl)
Copied to clipboard
Jun Gao, Yun Peng, Qian Qiao, Changhai Zhou, Yuhua Zhou, Shiyang Zhang, Shichao Weng, Zhenchang Xing, Xiaoxue Ren
| Challenge: | Existing code reasoning benchmarks evaluate final output correctness under a single implementation. |
| Approach: | They propose a Code Reasoning benchmark that evaluates code reasoning through implementation invariance and process transparency. |
| Outcome: | The proposed benchmarks lack implementation invariance and process transparency . they observe superficial execution where models arrive at correct outputs without reasoning . |
CoT-based Synthesizer: Enhancing LLM Performance through Answer Synthesis (2025.acl-long)
Copied to clipboard
| Challenge: | Existing inference scaling methods rely heavily on the quality of candidate responses . however, they are unable to produce correct answers when all candidates are flawed . |
| Approach: | They propose a CoT-based inference scaling strategy that leverages CoT reasoning to synthesize superior answers by analyzing complementary information from multiple candidate responses. |
| Outcome: | The proposed method improves performance on four benchmark datasets with seven policy models. |