Papers by Shima Imani
SymPyBench: A Dynamic Benchmark for Scientific Reasoning with Executable Python Code (2026.eacl-industry)
Copied to clipboard
| Challenge: | Existing benchmarks do not capture the complexity of structured, step-by-step reasoning essential in physics and related domains. |
| Approach: | They propose a large-scale synthetic benchmark of 15K university-level physics problems . they use structured, step-by-step reasoning and executable Python code to produce the ground-truth solution. |
| Outcome: | The proposed model is based on a set of 15K university-level physics problems with three question types. |
MathPrompter: Mathematical Reasoning using Large Language Models (2023.acl-industry)
Copied to clipboard
| Challenge: | Recent advances in natural language processing (NLP) can be attributed to massive scaling of Large Language Models (LLMs). |
| Approach: | They propose a technique that improves performance of Large Language Models (LLMs) on arithmetic problems along with increased reliance in the predictions. |
| Outcome: | The proposed technique improves performance on arithmetic problems and increases confidence in the output results. |
TRACE: A Framework for Analyzing and Enhancing Stepwise Reasoning in Vision-Language Models (2026.eacl-long)
Copied to clipboard
| Challenge: | Evaluating large vision-language models has focused on final-answer correctness, but this metric is often insufficient and misleading. |
| Approach: | They propose a framework that decomposes complex multimodal tasks into Auxiliary Reasoning Sets (ARS) ARS decomposition reveals how consistently a model reasons across sub-questions with structured dependencies. |
| Outcome: | a new framework improves diagnostic evaluation of large vision-language models . it decomposes complex multimodal tasks into auxiliary reasoning sets with structured dependencies . the framework pinpoints reasoning failures and exposes errors overlooked by standard evaluation . |