Papers by Pratyush Thakur
STEM-POM: Evaluating Language Models Math-Symbol Reasoning in Document Parsing (2025.findings-acl)
Copied to clipboard
| Challenge: | Advances in large language models have spurred research into enhancing their reasoning capabilities, particularly in math-rich STEM documents. |
| Approach: | They propose a benchmark dataset to evaluate LLMs’ reasoning abilities on math symbols within contextual scientific text. |
| Outcome: | The proposed dataset demonstrates that state-of-the-art LLMs achieve an average accuracy of 20-60% under in-context learning and 50-60% with fine-tuning, highlighting a substantial gap in their ability to classify mathematical symbols. |