Papers by Hassan Mansoor
Chart-based Reasoning: Transferring Capabilities from LLMs to VLMs (2024.findings-naacl)
Copied to clipboard
Victor Carbune, Hassan Mansoor, Fangyu Liu, Rahul Aralikatte, Gilles Baechler, Jindong Chen, Abhanshu Sharma
| Challenge: | Visual language models (VLMs) are achieving increasingly strong performance on multimodal tasks. |
| Approach: | They propose to transfer reasoning capabilities from large-language models to VLMs by constructing a 20x larger dataset and a larger dataset to improve general reasoning capabilities. |
| Outcome: | The proposed model outperforms larger models without an upstream OCR system while keeping inference time constant. |
LLMs cannot find reasoning errors, but can correct them given the error location (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent attempts to self-correct logical or reasoning errors often cause correct answers to become incorrect, resulting in poor performance overall. |
| Approach: | They propose to use a backtracking setup to test the correction abilities of LLMs on their mistake-finding ability to find logical mistakes. |
| Outcome: | The proposed model improves on 5 reasoning tasks, showing that it can correct logical mistakes without ground truth labels or training data. |
Self-play through Computational Runtimes improves Chart Reasoning (2025.findings-acl)
Copied to clipboard
| Challenge: | Vision-language models (VLMs) achieve impressive zero-shot performance on multimodal reasoning tasks. |
| Approach: | They propose a self-play programming interface which leverages visual reasoning models to generate code to decompose a complex visual reasoning task in sub-tasks and use it as a tool to solve decomposed tasks. |
| Outcome: | The proposed approach outperforms baselines on difficult chart reasoning benchmarks. |