Papers by Siddhartha Jain
Scaling Test-Time Compute to Achieve IOI Gold Medal with Open-Weight Models (2026.acl-long)
Copied to clipboard
Mehrzad Samadi, Aleksander Ficek, Sean Narenthiran, Siddhartha Jain, Wasi Uddin Ahmad, Somshubra Majumdar, Vahid Noroozi, Boris Ginsburg
| Challenge: | Competitive programming has become a rigorous benchmark for evaluating the reasoning and problem-solving capabilities of large language models (LLMs). |
| Approach: | They propose a scalable and reproducible test-time compute framework that achieves IOI gold-level performance using open-weight models. |
| Outcome: | The proposed framework achieves IOI gold-level performance using open-weight models . it scales consistently with available compute, narrowing the gap between open and closed systems. |
Lightweight reranking for language model generations (2024.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) can exhibit considerable variation in quality of sampled outputs. |
| Approach: | They propose a method for reranking LLM generations using pairwise statistics . they show strong improvements for selecting the best k generations for code generation tasks . |
| Outcome: | The proposed approach improves selection and generation quality for code generation tasks and autoformalization, summarization, and translation tasks. |
Reasoning in Token Economies: Budget-Aware Evaluation of LLM Reasoning Strategies (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing evaluations that focus on performance metrics miss a key factor: increased effectiveness due to additional compute. |
| Approach: | They propose to incorporate the compute budget into evaluations to provide a more informative comparison that takes into account both performance metrics and computational cost. |
| Outcome: | The proposed framework outperforms reasoning strategies when they use comparable compute resources. |