Papers by Hima Patel
STaD: Scaffolded Task Design for Identifying Compositional Skill Gaps in LLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks often treat complex tasks as monolithic, resulting in inconsistent performance and inconsistent explanations. |
| Approach: | They propose a framework for creating controlled variations of benchmark tasks based on the concept of scaffolding, which introduces structured, incremental support in a step-by-step manner. |
| Outcome: | The proposed framework enables systematic probing of model behavior by identifying the specific reasoning skill compositions they lack. |
Think Like You Execute: Verifiable Chain of Thought from Program Traces (2026.acl-industry)
Copied to clipboard
Shailja Thakur, Vaibhav Saxena, Rohan Kulkarni, Shivdeep Singh, Parameswaran Selvam, Hiroshi Kanayama, Hima Patel
| Challenge: | Current synthetic Chain-of-Thought (CoT) training data often consists of plausible-sounding explanations generated by teacher models, not verifiable accounts of actual program behavior. |
| Approach: | They propose to ground CoT generation directly in program execution traces to improve reasoning capabilities. |
| Outcome: | The proposed pipeline improves performance on live code benchmarks and on cruxEval-output and cruxeval-input. |