Chart-based Reasoning: Transferring Capabilities from LLMs to VLMs

Challenge:	Visual language models (VLMs) are achieving increasingly strong performance on multimodal tasks.
Approach:	They propose to transfer reasoning capabilities from large-language models to VLMs by constructing a 20x larger dataset and a larger dataset to improve general reasoning capabilities.
Outcome:	The proposed model outperforms larger models without an upstream OCR system while keeping inference time constant.

Similar Papers

Overview

Challenge

Approach

Outcome

Distill Visual Chart Reasoning Ability from LLMs to MLLMs (2025.findings-emnlp)

Copied to clipboard

Wei He, Zhiheng Xi, Wanxu Zhao, Xiaoran Fan, Yiwen Ding, Zifei Shan, Tao Gui, Qi Zhang, Xuanjing Huang

Language Models

Challenge:	a new method for generating chart annotations is proposed to improve visual reasoning in multimodal large language models.
Approach:	They propose a code-as-intermediary translation method for distilling visual reasoning abilities from LLMs to MLLMs.
Outcome:	The proposed method is cost-effective, efficient and scalable.

Are Large Vision Language Models up to the Challenge of Chart Comprehension and Reasoning (2024.findings-emnlp)

Copied to clipboard

Mohammed Saidul Islam, Raian Rahman, Ahmed Masry, Md Tahmid Rahman Laskar, Mir Tafseer Nayeem, Enamul Hoque

Multimodality

Challenge:	Recent studies have demonstrated that large vision language models (LVLMs) are not multi-modal and lack multi-tasking capabilities.
Approach:	They evaluate the performance of large vision language models (LVLMs) for chart understanding and reasoning tasks and compare them to open-source models.
Outcome:	The proposed models demonstrate impressive abilities in generating fluent texts covering high-level data insights, but they also encounter common problems like hallucinations, factual errors, and data bias.

Enhancing Advanced Visual Reasoning Ability of Large Language Models (2024.emnlp-main)

Copied to clipboard

Zhiyuan Li, Dongnan Liu, Chaoyi Zhang, Heng Wang, Tengfei Xue, Weidong Cai

Language Models

Challenge:	Recent advances in Vision-Language (VL) research have sparked new benchmarks for complex visual reasoning, challenging models’ advanced reasoning ability.
Approach:	They propose a novel multi-modal in-context learning methodology to enhance LLMs’ contextual understanding and reasoning.
Outcome:	The proposed model achieves SOTA performance among all visual reasoning tasks and achieves a 'higher level of accuracy' than previous models.

ChartCoder: Advancing Multimodal Large Language Model for Chart-to-Code Generation (2025.acl-long)

Copied to clipboard

Xuanle Zhao, Xianzhen Luo, Qi Shi, Chi Chen, Shuo Wang, Zhiyuan Liu, Maosong Sun

Multimodality

Challenge:	Existing open-source MLLMs fail to fully capture dense information embedded in charts . current models still face significant challenges in understanding and analyzing visual tasks such as captioning and question answering.
Approach:	They propose a chart-to-code MLLM which leverages Code LLMs as the language backbone to enhance the executability of the generated code.
Outcome:	The proposed model surpasses existing open-source models on chart-to-code benchmarks with only 7B parameters and provides lossless representations that contain all critical details.

MuSe: Multi-Stage Graph Reasoning via Vision-Language Models (2026.acl-long)

Copied to clipboard

Guanyu Wang, Xu Chu, Zhijie Tan, Xinrong Chen, Tong Mo, Weiping Li

Multimodality

Challenge:	Graph Neural Networks (GNNs) and graph transformers are inadequate for tasks with limited generalization.
Approach:	They propose a multi-stage graph reasoning framework based on vision-language models that incrementally samples and visualizes task-relevant subgraphs.
Outcome:	The proposed framework outperforms existing benchmarks in Graph-related tasks.

ChartInsights: Evaluating Multimodal Large Language Models for Low-Level Chart Question Answering (2024.findings-emnlp)

Copied to clipboard

Yifan Wu, Lutao Yan, Leixian Shen, Yunhai Wang, Nan Tang, Yuyu Luo

Language Models

Challenge:	Chart question answering (ChartQA) tasks are a critical part of visualization charts.
Approach:	They propose a chart question answering task that uses MLLMs to analyze charts . they propose 'Chain-of-Charts' textual prompt strategy that directs attention to visual elements .
Outcome:	The proposed model improves performance by 14.41% and 80% in low-level ChartQA tasks.

Mapping the Minds of LLMs: A Graph-Based Analysis of Reasoning LLMs (2025.emnlp-main)

Copied to clipboard

Zhen Xiong, Yujun Cai, Zhecheng Li, Yiwei Wang

Responsible & Trustworthy NLP

Challenge:	Large Reasoning Models (LRMs) often display unstable behaviors, e.g., hallucinating unsupported premises, overthinking simple tasks, and displaying higher sensitivity to prompt variations.
Approach:	They propose a graph-based analytical framework that clusters long, verbose CoT outputs into semantically coherent reasoning steps, then constructs directed reasoning graphs to capture contextual and logical dependencies among these steps.
Outcome:	The proposed framework enables quantitative evaluation of internal reasoning structure and quality beyond conventional metrics and provides practical insights for prompt engineering and cognitive analysis of LLMs.

Advancement in Graph Understanding: A Multimodal Benchmark and Fine-Tuning of Vision-Language Models (2024.acl-long)

Copied to clipboard

Qihang Ai, Jiafan Li, Jincheng Dai, Jianwu Zhou, Lemao Liu, Haiyun Jiang, Shuming Shi

Multimodality

Challenge:	Graph data organizes complex relationships and interactions between objects . Graph neural networks (GNNs) are becoming more popular in graph learning .
Approach:	They propose a new paradigm for interactive and instructional graph data understanding and reasoning . they first evaluate the capabilities of public VLMs in graph learning from multiple aspects .
Outcome:	The proposed model achieves an accuracy increase of 5%-15% compared to baseline models . the best-performing model achieve scores comparable to Gemini in GPT-asissted Evaluation .

ChartM3: A Multi-Stage Code-Driven Pipeline for Constructing Multi-Dimensional and Multi-Step Visual Reasoning Data in Chart Comprehension (2025.findings-emnlp)

Copied to clipboard

Duo Xu, Hao Cheng, Xin Lin, Zhen Xie, Hao Henry Wang

Multimodality

Challenge:	Currently, research on complex chart understanding tasks is limited . a pipeline for visual reasoning datasets addresses these limitations .
Approach:	They propose a code-driven pipeline for generating visual reasoning datasets . pipeline integrates retrieval-augmented generation to retrieve professional chart templates .
Outcome:	The proposed pipeline enhances chart diversity and data quality through model-based evaluation.

MultiChartQA: Benchmarking Vision-Language Models on Multi-Chart Problems (2025.naacl-long)

Copied to clipboard

Zifeng Zhu, Mengzhao Jia, Zhihan Zhang, Lang Li, Meng Jiang

Multimodality

Challenge:	Existing chart understanding benchmarks focus on single-chart tasks, neglecting multi-hop reasoning required to extract and integrate information from multiple charts.
Approach:	They propose a benchmark that evaluates MLLMs’ capabilities in four key areas: direct question answering, parallel question answering and comparative reasoning.
Outcome:	The proposed benchmark evaluates MLLMs’ capabilities in four key areas: direct question answering, parallel question answering and comparative reasoning.