POLYCHARTQA: Benchmarking Large Vision-Language Models with Multilingual Chart Question Answering (2026.acl-long)
Copied to clipboard
| Challenge: | Existing chart understanding benchmarks are overwhelmingly English-centric, limiting their accessibility and relevance to global audiences. |
| Approach: | They propose a multilingual chart question answering benchmark that enables efficient multilingual generation via data translation and code reuse. |
| Outcome: | The proposed benchmark systematically evaluates multilingual chart understanding on state-of-the-art LVLMs and shows a significant performance gap between English and other languages. |
Similar Papers
Beyond Single Plots: A Benchmark for Question Answering on Multi-Charts (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing research on chart understanding has been limited to single chart images. |
| Approach: | They propose a dataset specifically designed for question answering over multi-chart images. |
| Outcome: | The proposed method shows a 27.4% LLM-based accuracy drop on human-authored questions and a 5.39% gain in the human-generated questions. |
MultiChartQA: Benchmarking Vision-Language Models on Multi-Chart Problems (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing chart understanding benchmarks focus on single-chart tasks, neglecting multi-hop reasoning required to extract and integrate information from multiple charts. |
| Approach: | They propose a benchmark that evaluates MLLMs’ capabilities in four key areas: direct question answering, parallel question answering and comparative reasoning. |
| Outcome: | The proposed benchmark evaluates MLLMs’ capabilities in four key areas: direct question answering, parallel question answering and comparative reasoning. |
ChartQAPro: A More Diverse and Challenging Benchmark for Chart Question Answering (2025.findings-acl)
Copied to clipboard
Ahmed Masry, Mohammed Saidul Islam, Mahir Ahmed, Aayush Bajaj, Firoz Kabir, Aaryaman Kartha, Md Tahmid Rahman Laskar, Mizanur Rahman, Shadikur Rahman, Mehrad Shahmohammadi, Megh Thakkar, Md Rizwan Parvez, Enamul Hoque, Shafiq Joty
| Challenge: | Chart Question Answering systems are limited in their ability to interpret data visually and reason with visual representations. |
| Approach: | They propose a chart-based chart question-answering system that includes 1,341 charts from 99 diverse sources and 1,948 questions in various types. |
| Outcome: | The new benchmark includes 1,341 charts from 99 diverse sources and 1,948 questions in various types. |
WorldCuisines: A Massive-Scale Benchmark for Multilingual and Multicultural Visual Question Answering on Global Cuisines (2025.naacl-long)
Copied to clipboard
Genta Indra Winata, Frederikus Hudi, Patrick Amadeus Irawan, David Anugraha, Rifki Afina Putri, Wang Yutong, Adam Nohejl, Ubaidillah Ariq Prathama, Nedjma Ousidhoum, Afifa Amriani, Anar Rzayev, Anirban Das, Ashmari Pramodya, Aulia Adila, Bryan Wilie, Candy Olivia Mawalim, Cheng Ching Lam, Daud Abolade, Emmanuele Chersoni, Enrico Santus, Fariz Ikhwantri, Garry Kuwanto, Hanyang Zhao, Haryo Akbarianto Wibowo, Holy Lovenia, Jan Christian Blaise Cruz, Jan Wira Gotama Putra, Junho Myung, Lucky Susanto, Maria Angelica Riera Machin, Marina Zhukova, Michael Anugraha, Muhammad Farid Adilazuarda, Natasha Christabelle Santosa, Peerat Limkonchotiwat, Raj Dabre, Rio Alexander Audino, Samuel Cahyawijaya, Shi-Xiong Zhang, Stephanie Yulia Salim, Yi Zhou, Yinxuan Gui, David Ifeoluwa Adelani, En-Shiun Annie Lee, Shogo Okada, Ayu Purwarianti, Alham Fikri Aji, Taro Watanabe, Derry Tanti Wijaya, Alice Oh, Chong-Wah Ngo
| Challenge: | Vision Language Models struggle with cultural-specific knowledge, especially in languages other than English and in underrepresented cultural contexts. |
| Approach: | They propose a visual question answering (VQA) dataset with text-image pairs across 30 languages and dialects and a training dataset. |
| Outcome: | The proposed model performs better with correct location context, but struggles with adversarial contexts and predicting specific regional cuisines and languages. |
MKQA: A Linguistically Diverse Benchmark for Multilingual Open Domain Question Answering (2021.tacl-1)
Copied to clipboard
| Challenge: | Existing multilingual QA datasets lack linguistic diversity and comparable evaluation between languages. |
| Approach: | They propose a multilingual question-answer evaluation set with 10k English queries and human translations of them into 25 additional languages and dialects. |
| Outcome: | The proposed model is based on a multilingual knowledge questions and answers evaluation set with 26 languages. |
MTVQA: Benchmarking Multilingual Text-Centric Visual Question Answering (2025.findings-acl)
Copied to clipboard
Jingqun Tang, Qi Liu, Yongjie Ye, Jinghui Lu, Shu Wei, An-Lan Wang, Chunhui Lin, Hao Feng, Zhen Zhao, Yanjie Wang, Yuliang Liu, Hao Liu, Xiang Bai, Can Huang
| Challenge: | Text-Centric Visual Question Answering (TEC-VQA) is a text-centric visual task understanding tool. |
| Approach: | They introduce a benchmark that features human expert annotations across 9 languages . they prioritize the text in question-answer pairs while disregarding visual text in images . |
| Outcome: | The proposed benchmarks prioritize the text in question-answer pairs while disregarding visual text in images. |
Are Large Vision Language Models up to the Challenge of Chart Comprehension and Reasoning (2024.findings-emnlp)
Copied to clipboard
Mohammed Saidul Islam, Raian Rahman, Ahmed Masry, Md Tahmid Rahman Laskar, Mir Tafseer Nayeem, Enamul Hoque
| Challenge: | Recent studies have demonstrated that large vision language models (LVLMs) are not multi-modal and lack multi-tasking capabilities. |
| Approach: | They evaluate the performance of large vision language models (LVLMs) for chart understanding and reasoning tasks and compare them to open-source models. |
| Outcome: | The proposed models demonstrate impressive abilities in generating fluent texts covering high-level data insights, but they also encounter common problems like hallucinations, factual errors, and data bias. |
MULTITAT: Benchmarking Multilingual Table-and-Text Question Answering (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing TATQA datasets are limited to English, leading to drawbacks . existing datasets overlook challenges of multilingual TAT-QA and do not reflect real-world multilingual scenarios . |
| Approach: | They propose a multilingual TATQA dataset that can be translated into 10 languages . they use data from 3 mainstream TATQ datasets and analyze the results . |
| Outcome: | The proposed dataset outperforms other baselines by an average of 3.3 . |
ChartMind: A Comprehensive Benchmark for Complex Real-world Multimodal Chart Question Answering (2025.emnlp-main)
Copied to clipboard
| Challenge: | Chart question answering (CQA) is a multimodal task for evaluating the reasoning capabilities of vision-language models. |
| Approach: | They propose a chart question answering benchmark that incorporates multilingual contexts and supports open-domain textual outputs. |
| Outcome: | The proposed framework outperforms the previous three common CQA paradigms: instruction-following, OCR-enhanced, and chain-of-thought. |
XLQA: A Benchmark for Locale-Aware Multilingual Open-Domain Question Answering (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown significant progress in Open-domain question answering (ODQA) but most evaluations focus on English and assume locale-invariant answers across languages. |
| Approach: | They propose a benchmark specifically designed for locale-sensitive multilingual ODQA that uses 3,000 English seed questions expanded to eight languages. |
| Outcome: | The proposed benchmarks are based on 3,000 English seed questions expanded to eight languages and a human-verified annotation distinguishing locale-invariant and locale-sensitive cases. |