Code-SPA: Style Preference Alignment to Large Language Models for Effective and Robust Code Debugging (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated impressive capabilities in coding tasks like code generation and debugging. |
| Approach: | They propose a method which aligns noisy code with the well-structured style familiar to LLMs, mitigating the impact of stylistic inconsistencies. |
| Outcome: | The proposed method improves debugging performance on poorly styled code across the HumanEval, MBPP and EvalPlus datasets. |
Similar Papers
Teaching Your Models to Understand Code via Focal Preference Alignment (2025.emnlp-main)
Copied to clipboard
Jie Wu, Haoling Li, Xin Zhang, Xiao Liu, Yangyu Huang, Jianwen Luo, Yizhen Zhang, Zuchao Li, Ruihang Chu, Yujiu Yang, Scarlett Li
| Challenge: | Existing methods for supervised fine-tuning focus on unit test feedback to construct preference pairs. |
| Approach: | They propose a preference alignment framework that mimics human iterative debugging to refine Code LLMs. |
| Outcome: | Experiments show that Preference Learning improves on BigCodeBench and BigCodeBind tasks. |
code-transformed: The Influence of Large Language Models on Code (2026.findings-eacl)
Copied to clipboard
| Challenge: | Using Large Language Models, code generation capabilities have transformed programming practices. |
| Approach: | They analyze 20,000 GitHub repositories linked to arXiv papers published between 2020 and 2025 . they identify measurable trends in the evolution of coding style that align with LLM-generated code . |
| Outcome: | The proposed study examines 20,000 GitHub repositories linked to arXiv papers . it finds that LLMs influence code style, and that they can be observed in real-world code . |
Grammar-Based Code Representation: Is It a Worthy Pursuit for LLMs? (2025.findings-acl)
Copied to clipboard
Qingyuan Liang, Zhao Zhang, Zeyu Sun, Zheng Lin, Qi Luo, Xiao Yueyi, Yizhou Chen, Yuqun Zhang, Haotian Zhang, Lu Zhang, Chenbin Chenbin, Yingfei Xiong
| Challenge: | Existing research demonstrates the effectiveness of grammar-based code representations in small-scale models, showing their ability to reduce syntax errors and enhance performance. |
| Approach: | They develop a series of billion-scale grammar-based code representations that incorporate grammar rules into the code generation process. |
| Outcome: | Experiments on HumanEval and MBPP show that grammar-based representations reduce syntax errors and improve performance even in billion-scale models. |
CodeArena: Evaluating and Aligning CodeLLMs on Human Preference (2025.emnlp-main)
Copied to clipboard
Jian Yang, Jiaxi Yang, Wei Zhang, Jin Ke, Yibo Miao, Lei Zhang, Liqun Yang, Zeyu Cui, Yichang Zhang, Zhoujun Li, Binyuan Hui, Junyang Lin
| Challenge: | Code large language models (codeLLMs) focus on synthesizing the correct code snippet, ignoring the alignment with human preferences. |
| Approach: | They propose a benchmark code-based on 40 categories and 44 programming languages to emulate real-world coding tasks. |
| Outcome: | The proposed benchmarks show that open-source code LLMs perform better than open-sourced ones. |
MdEval: Massively Multilingual Code Debugging (2026.findings-acl)
Copied to clipboard
Shukai Liu, Linzheng Chai, Jian Yang, Jiajun Shi, He Zhu, Liran Wang, Jin Ke, Wei Zhang, Hualei Zhu, Shuyue Guo, Tao Sun, Jiaheng Liu, Yunlong Duan, Yu Hao, Liqun Yang, Guanglin Niu, Ge Zhang, Zhoujun Li
| Challenge: | Existing benchmarks primarily focus on Python and are limited in terms of language diversity. |
| Approach: | They propose a multilingual debugging benchmark that includes 3.9K test samples of 20 programming languages and introduces the debug instruction corpora MdEval-Instruct by injecting bugs into the correct multilingual queries and solutions. |
| Outcome: | The proposed benchmark includes 3.9K test samples of 20 programming languages and covers the automated program repair task, bug localization task, and bug identification task. |
PythonSaga: Redefining the Benchmark to Evaluate Code Generating LLMs (2024.findings-emnlp)
Copied to clipboard
| Challenge: | *HumanEval* and *MBPP* are two popular benchmarks for Python code generation. |
| Approach: | They propose a large-scale human evaluation of two popular Python benchmarks . they propose 185 hand-crafted prompts in a balanced representation of 38 programming concepts across diverse difficulty levels. |
| Outcome: | The proposed benchmarks show a critical bias towards a limited set of programming concepts, neglecting most of the other concepts entirely. |
Alignment with Fill-In-the-Middle for Enhancing Code Generation (2025.emnlp-main)
Copied to clipboard
Houxing Ren, Zimu Lu, Weikang Shi, Haotian Hou, Yunqiao Yang, Ke Wang, Aojun Zhou, Junting Pan, Mingjie Zhan, Hongsheng Li
| Challenge: | Existing methods for generating test cases with limited training data are not reliable and may be counterproductive. |
| Approach: | They propose a method that splits code snippets into smaller, granular blocks, creating more diverse DPO pairs from the same test cases. |
| Outcome: | The proposed approach shows significant improvements in code generation tasks on benchmark datasets such as HumanEval (+), MBPP (+), and APPS. |
CodeT5+: Open Code Large Language Models for Code Understanding and Generation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing code LLMs adopt a specific architecture or rely on a unified encoder-decoder network for downstream tasks, lacking flexibility to operate in the optimal architecture for a particular task. |
| Approach: | They propose to initialize code LLMs with frozen off-the-shelf LLM and explore instruction-tuning to align with natural language instructions. |
| Outcome: | The proposed model outperforms open-source LLMs on 20 code-related benchmarks. |
RoCoIns: Enhancing Robustness of Large Language Models through Code-Style Instructions (2024.lrec-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown remarkable capabilities in following human instructions and solving NLU tasks. |
| Approach: | They propose to use code style instructions to replace typically natural language instructions to provide more precise instructions and strengthen the robustness of LLMs. |
| Outcome: | The proposed method outperforms natural language models on eight robustness datasets and achieves an improvement of 5.68% in test set accuracy and a reduction of 5.66 points in Attack Success Rate (ASR). |
ExeCoder: Empowering Large Language Models with Executability Representation for Code Translation (2025.emnlp-main)
Copied to clipboard
Minghua He, Yue Chen, Fangkai Yang, Pu Zhao, Wenjie Yin, Yu Kang, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang
| Challenge: | Existing code translation models only learn the contextual semantics of code during pre-training, neglecting executability information closely related to the execution state of the code. |
| Approach: | They propose an LLM specifically designed for code translation called ExeCoder . it uses executability representations such as functional semantics and syntax structures to enhance LLMs' capabilities. |
| Outcome: | The proposed model outperforms existing open-source code translation models on two metrics. |