Papers with TNR
CodeContests-O: Powering LLMs via Feedback-Driven Iterative Test Case Generation (2026.findings-acl)
Copied to clipboard
Jianfeng Cai, Jinhua Zhu, Ruopei Sun, Kangwen Zhao, Dongyun Xue, Mingxiao Feng, Wengang Zhou, Houqiang Li
| Challenge: | Existing approaches to synthesize test cases using Large Language Models (LLMs) rely on the model’s intrinsic generation capabilities without external feedback, resulting in insufficiently diverse cases. |
| Approach: | They propose a feedback-driven iterative framework that leverages Large Language Models to generate initial test cases, execute them against known correct and incorrect solutions, and utilizes the failed results as feedback to guide the LLM in refining the test cases toward high fidelity and discriminability. |
| Outcome: | The proposed method outperforms the existing codecontests and codecontests+ models by 4.30% and 8.78%. |
CodeHacker: Automated Test Case Generation for Detecting Vulnerabilities in Competitive Programming Solutions (2026.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks for Large Language Models often lack coverage for subtle corner cases . a substantial amount of effort has been applied to address this challenge . |
| Approach: | They propose a framework that generates adversarial test cases that expose latent vulnerabilities in code submissions. |
| Outcome: | The proposed framework improves the True Negative Rate (TNR) of existing datasets and generates superior adversarial cases on liveCodeBench. |
Beyond Timestamps: Bridging Forward and Backward Reasoning in Temporal Numerical and Relational Understanding (2026.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks for Temporal Numerical and Relational reasoning rely on single-task evaluation paradigms. |
| Approach: | They propose a benchmark to evaluate Temporal Numerical and Relational reasoning . they propose QA and verification, and a Consistency Rate to quantify robustness . |
| Outcome: | The proposed framework evaluates both Temporal Numerical and Relational reasoning . it measures the alignment between QA and FV and the Consistency Rate measures robustness across these directions. |