Papers by Andrew Head
Scaling Text-Rich Image Understanding via Code-Guided Synthetic Multimodal Data Generation (2025.acl-long)
Copied to clipboard
Yue Yang, Ajay Patel, Matt Deitke, Tanmay Gupta, Luca Weihs, Andrew Head, Mark Yatskar, Chris Callison-Burch, Ranjay Krishna, Aniruddha Kembhavi, Christopher Clark
| Challenge: | Vision-language models struggle to understand text-rich images due to the scarcity of diverse text-only large language data. |
| Approach: | They propose a framework that leverages the coding capabilities of text-only large language models to create synthetic text-rich multimodal data. |
| Outcome: | The proposed framework can generate high-quality instruction-tuning data using Python, HTML, LaTeX and other languages. |
Modeling Mathematical Notation Semantics in Academic Papers (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models that can predict mathematical notation are unable to analyze mathematical notations reliably. |
| Approach: | They propose two tasks that can be used to train a model that selectively masks notation tokens and encodes left and/or right sentences as context. |
| Outcome: | The proposed model performs better than baseline models trained by masked language modeling compared to baseline models, but is less accurate than token-level models . |