KodCode: A Diverse, Challenging, and Verifiable Synthetic Dataset for Coding (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing code-focused resources typically fail to ensure either the breadth of coverage or verifiable correctness. |
| Approach: | They propose a synthetic dataset that provides high-quality, verifiable training data for Large Language Models for coding. |
| Outcome: | The proposed dataset surpasses Qwen2.5-Coder-32B-Instruct and DeepSeek-R1-Distill-Llama-70B in performance on coding benchmarks. |
Similar Papers
PuzzleClone: A DSL-Powered Framework for Synthesizing Verifiable Data (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing datasets with verifiable answers are limited in reliability, diversity, and scalability . a new approach to generate verifikatable data at scale is needed to improve models' performance . |
| Approach: | They propose a formal framework for synthesizing verifiable data at scale using a novel DSL-driven approach. |
| Outcome: | The proposed framework improves performance on a wide range of puzzles and logic benchmarks. |
UnitCoder: Scalable Code Synthesis from Pre-training Corpora (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) excel at code understanding and generation, yet code generation remains a challenge. |
| Approach: | They propose a model that supervises pre-training data quality through automatically generated unit tests while ensuring correctness via an iterative fix and refine flow. |
| Outcome: | The proposed model improves performance on a large dataset with high quality pre-training data. |
CodeContests-O: Powering LLMs via Feedback-Driven Iterative Test Case Generation (2026.findings-acl)
Copied to clipboard
Jianfeng Cai, Jinhua Zhu, Ruopei Sun, Kangwen Zhao, Dongyun Xue, Mingxiao Feng, Wengang Zhou, Houqiang Li
| Challenge: | Existing approaches to synthesize test cases using Large Language Models (LLMs) rely on the model’s intrinsic generation capabilities without external feedback, resulting in insufficiently diverse cases. |
| Approach: | They propose a feedback-driven iterative framework that leverages Large Language Models to generate initial test cases, execute them against known correct and incorrect solutions, and utilizes the failed results as feedback to guide the LLM in refining the test cases toward high fidelity and discriminability. |
| Outcome: | The proposed method outperforms the existing codecontests and codecontests+ models by 4.30% and 8.78%. |
CodeEvo: Interaction-Driven Synthesis of Code-centric Data through Hybrid and Iterative Feedback (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for generating instruction-code pairs rely on rigid heuristics and are labor-intensive. |
| Approach: | They propose a dual-agent architecture that integrates a Coder and a Reviewer to orchestrate the generation trajectory. |
| Outcome: | The proposed architecture outperforms baselines on a large-scale dataset of instruction-code pairs with stepped difficulty levels. |
ACECODER: Acing Coder RL via Automated Test-Case Synthesis (2025.acl-long)
Copied to clipboard
| Challenge: | Recent coder models have been driven by supervised fine-tuning, but the potential of reinforcement learning remains unexplored due to the lack of reliable reward data/model in the code domain. |
| Approach: | They propose a pipeline that generates extensive test-case pairs from existing code data and constructs preference pairs based on pass rates over sampled programs. |
| Outcome: | The proposed pipeline generates extensive (question, test-cases) pairs from existing code data and trains them with Bradley-Terry loss. |
Case2Code: Scalable Synthetic Data for Code Generation (2025.coling-main)
Copied to clipboard
Yunfan Shao, Linyang Li, Yichuan Ma, Peiji Li, Demin Song, Qinyuan Cheng, Shimin Li, Xiaonan Li, Pengyu Wang, Qipeng Guo, Hang Yan, Xipeng Qiu, Xuanjing Huang, Dahua Lin
| Challenge: | Large Language Models (LLMs) have shown outstanding breakthroughs in code generation. |
| Approach: | They propose a case-to-code induction task that exploits the expressiveness and correctness of programs by incorporating LLMs into their training. |
| Outcome: | The proposed task improves distribution case-to-code induction and various coding generation tasks. |
Seed-Free Synthetic Data Generation Framework for Instruction-Tuning LLMs: A Case Study in Thai (2024.acl-srw)
Copied to clipboard
| Challenge: | Xue et al., 2024) have demonstrated that large language models can perform at human level across multitudes of tasks and domains. |
| Approach: | They propose a seed-free framework for generating synthetic instruction-tuning data that incorporates fluency, diversity, and cultural context. |
| Outcome: | The proposed framework achieves competitive performance using only 5,000 instructions compared to state-of-the-art Thai LLMs trained on hundreds of thousands of instructions. |
Powering Verifiable Learning via Automated Evolutionary Data Synthesis (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to building generalizable verifiable data are task-specific and lack a principled, universal evaluator of verifikatability. |
| Approach: | They propose a task-agnostic, strategy-guided, executably-checkable data synthesis framework that synthesizes problems, diverse candidate solutions and verification artifacts from a single source. |
| Outcome: | The proposed framework synthesizes problems, candidates, and verification artifacts from human-annotated and strategy-induced checks and iteratively discovers strategies. |
Evaluating Language Models as Synthetic Data Generators (2025.acl-long)
Copied to clipboard
Seungone Kim, Juyoung Suk, Xiang Yue, Vijay Viswanathan, Seongyun Lee, Yizhong Wang, Kiril Gashteovski, Carolin Lawrence, Sean Welleck, Graham Neubig
| Challenge: | Prior studies have focused on developing effective data generation methods, but lack systematic comparison of different LMs as data generators in a unified setting. |
| Approach: | They propose to use a benchmark to compare language models' data generation abilities against a set of standardized settings and metrics. |
| Outcome: | The proposed benchmark provides standardized settings and metrics to evaluate LMs’ data generation abilities. |
Let’s Synthesize Step by Step: Iterative Dataset Synthesis with Large Language Models by Extrapolating Errors from Small Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | *Data Synthesis* is a promising way to train a small model with very little labeled data. |
| Approach: | They propose a framework that iteratively extrapolates the errors of a small model trained on a real-world validation dataset using a large language model. |
| Outcome: | The proposed framework reduces the gap between the synthesized dataset and the real data . it improves on multiple NLP tasks and on large models with human-annotated data. |