Papers with Charts
Judging the Judges: Can Large Vision-Language Models Fairly Evaluate Chart Comprehension and Reasoning? (2025.acl-industry)
Copied to clipboard
Md Tahmid Rahman Laskar, Mohammed Saidul Islam, Ridwan Mahbub, Ahmed Masry, Mizanur Rahman, Amran Bhuiyan, Mir Tafseer Nayeem, Shafiq Joty, Enamul Hoque, Jimmy Huang
| Challenge: | Large Vision-Language Models (LVLMs) are expensive and time-consuming to evaluate . however, they are limited in their use in industrial settings due to their limited availability and limited resources. |
| Approach: | They evaluate 13 open-source LVLMs as judges for diverse chart comprehension and reasoning tasks. |
| Outcome: | The proposed models can be used to assess chart comprehension and reasoning tasks, but they are expensive and time-consuming. |
TinyChart: Efficient Chart Understanding with Program-of-Thoughts Learning and Visual Token Merging (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have shown that multimodal large language models can be useful for chart understanding, but their size limits their use in resource-constrained environments. |
| Approach: | They propose an efficient multimodal large language model with only 3B parameters for chart understanding. |
| Outcome: | The proposed model outperforms several chart-understanding MLLMs with up to 13B parameters on ChartQA, Chart-to-Text, Chart to Table, OpenCQA, and ChartX. |
ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing datasets that focus on complex reasoning questions do not address such questions as they are template-based and answers come from a fixed-vocabulary. |
| Approach: | They propose a large-scale benchmark that uses visual and logical reasoning to answer questions using a transformer-based model. |
| Outcome: | The proposed models achieve state-of-the-art on the previous datasets and on the current one, but also show that they have several challenges in answering complex reasoning questions. |
Chart-to-Text: A Large-Scale Benchmark for Chart Summarization (2022.acl-long)
Copied to clipboard
Shankar Kantharaj, Rixie Tiffany Leong, Xiang Lin, Ahmed Masry, Megh Thakkar, Enamul Hoque, Shafiq Joty
| Challenge: | Inferring key insights from charts can be challenging and time-consuming. |
| Approach: | They propose a task where the goal is to explain a chart and summarize key takeaways from it in natural language. |
| Outcome: | The proposed model produces fluent summaries but suffers from hallucinations and factual errors . the proposed model is compared with other models and can be used to generate BLEU scores . |
ChartAssistant: A Universal Chart Multimodal Language Model via Chart-to-Table Pre-training and Multitask Instruction Tuning (2024.findings-acl)
Copied to clipboard
| Challenge: | Charts are an effective tool for understanding data patterns, but their combination of graphical elements and textual components poses challenges for general-purpose multimodal models. |
| Approach: | They propose a chart-based vision-language model for universal chart comprehension and reasoning that leverages a dataset of chart-related tasks. |
| Outcome: | The proposed model outperforms the state-of-the-art charts with zero-shot setting on various chart tasks. |
ChartGaze: Enhancing Chart Understanding in LVLMs with Eye-Tracking Guided Attention Refinement (2025.emnlp-main)
Copied to clipboard
Ali Salamatian, Amirhossein Abaskohi, Wan-Cyuan Fan, Mir Rayat Imtiaz Hossain, Leonid Sigal, Giuseppe Carenini
| Challenge: | Chart question answering (CQA) is a key research challenge for large vision-language models . recent efforts focus on leveraging LVLMs directly on chart images . |
| Approach: | They propose a gaze-guided attention refinement that aligns image-text attention with human fixations to improve chart reasoning quality and interpretability. |
| Outcome: | The proposed approach improves answer accuracy and attention alignment yielding gains of up to 2.56 percentage points across multiple models. |
ChartInstruct: Instruction Tuning for Chart Comprehension and Reasoning (2024.findings-acl)
Copied to clipboard
| Challenge: | Charts provide visual representations of data and are used for analyzing information, addressing queries, and conveying insights to others. |
| Approach: | They propose a chart-specific vision-language Instruction-following dataset with 191K instructions and a pipeline model that extracts chart data tables and inputs them into a LLM. |
| Outcome: | The proposed model can solve a wide range of chart-related tasks, achieving state-of-the-art results on four tasks. |
OpenCQA: Open-ended Question Answering with Charts (2022.emnlp-main)
Copied to clipboard
| Challenge: | OpenCQA is a task to answer open-ended questions about charts with descriptive texts. |
| Approach: | They propose a task to answer open-ended questions about charts with descriptive texts. |
| Outcome: | The proposed task is to answer an open-ended question about a chart with descriptive texts. |
UniChart: A Universal Vision-language Pretrained Model for Chart Comprehension and Reasoning (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for chart-based data analysis neglect explicit modeling of chart structures. |
| Approach: | They propose a pretrained model for chart comprehension and reasoning that encodes relevant text, data, and visual elements of charts and uses a chart-grounded text decoder for text generation. |
| Outcome: | The proposed model outperforms existing methods that lack explicit modeling of chart structures and lacks explicit modeling. |
ChartQAPro: A More Diverse and Challenging Benchmark for Chart Question Answering (2025.findings-acl)
Copied to clipboard
Ahmed Masry, Mohammed Saidul Islam, Mahir Ahmed, Aayush Bajaj, Firoz Kabir, Aaryaman Kartha, Md Tahmid Rahman Laskar, Mizanur Rahman, Shadikur Rahman, Mehrad Shahmohammadi, Megh Thakkar, Md Rizwan Parvez, Enamul Hoque, Shafiq Joty
| Challenge: | Chart Question Answering systems are limited in their ability to interpret data visually and reason with visual representations. |
| Approach: | They propose a chart-based chart question-answering system that includes 1,341 charts from 99 diverse sources and 1,948 questions in various types. |
| Outcome: | The new benchmark includes 1,341 charts from 99 diverse sources and 1,948 questions in various types. |
ChartLens: Fine-grained Visual Attribution in Charts (2025.acl-long)
Copied to clipboard
| Challenge: | MLLMs suffer from hallucinations, where generated text fails to align with visual inputs. |
| Approach: | They propose a chart attribution algorithm that uses segmentation-based techniques to identify chart objects and employs set-of-marks prompting with MLLMs for fine-grained visual attribution. |
| Outcome: | The proposed algorithm improves fine-grained attributions by 26-66% . |
From Charts to Fair Narratives: Uncovering and Mitigating Geo-Economic Biases in Chart-to-Text (2025.emnlp-main)
Copied to clipboard
Ridwan Mahbub, Mohammed Saidul Islam, Mir Tafseer Nayeem, Md Tahmid Rahman Laskar, Mizanur Rahman, Shafiq Joty, Enamul Hoque
| Challenge: | Existing VLMs produce more positive descriptions for high-income countries compared to middle- or low-income nations, even when country attribution is the only variable changed. |
| Approach: | They propose to automate the process by generating textual summaries of charts using vision-language models to understand how a country’s economic status influences the sentiment of generated summary. |
| Outcome: | The proposed model amplifys geo-economic biases in 6,000 chart-country pairs from six widely used vision-language models to understand how a country’s economic status influences the sentiment of generated summaries. |
Beyond Single Plots: A Benchmark for Question Answering on Multi-Charts (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing research on chart understanding has been limited to single chart images. |
| Approach: | They propose a dataset specifically designed for question answering over multi-chart images. |
| Outcome: | The proposed method shows a 27.4% LLM-based accuracy drop on human-authored questions and a 5.39% gain in the human-generated questions. |
POLYCHARTQA: Benchmarking Large Vision-Language Models with Multilingual Chart Question Answering (2026.acl-long)
Copied to clipboard
| Challenge: | Existing chart understanding benchmarks are overwhelmingly English-centric, limiting their accessibility and relevance to global audiences. |
| Approach: | They propose a multilingual chart question answering benchmark that enables efficient multilingual generation via data translation and code reuse. |
| Outcome: | The proposed benchmark systematically evaluates multilingual chart understanding on state-of-the-art LVLMs and shows a significant performance gap between English and other languages. |