Papers by Tianran Sun
Building A Proof-Oriented Programmer That Is 64% Better Than GPT-4o Under Data Scarcity (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing proof-oriented programming languages struggle due to data scarcity . existing proof-based models struggle due a lack of sufficient corpora for proof-orientated programming languages such as F* . |
| Approach: | They propose a method that synthesizes proof-oriented programming problems for proficiency in a language and incorporates diverse coding data for reasoning capability elicitation. |
| Outcome: | The proposed method outperforms existing proof-oriented models in function- and repository-level proof-based programming by 64% relative margin and improves GPT-4o's performance by 54% by repairing outputs over GPT-4) |
LastingBench: Defend Benchmarks Against Knowledge Leakage (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to detect and safeguard LLMs against knowledge leakage fail to address the long-term challenge of mitigating it. |
| Approach: | They propose a method to reinforce and safeguard existing benchmarks against knowledge leakage by perturbation-based detection and counterfactual rewriting to disrupt memorization while preserving original intent. |
| Outcome: | The proposed method reduces memorization effects in long-context QA benchmarks, providing a more accurate assessment of model reasoning and generalization abilities. |