MatPlotAgent: Method and Evaluation for LLM-Based Agentic Scientific Data Visualization (2024.findings-acl)
Copied to clipboard
Zhiyu Yang, Zihan Zhou, Shuo Wang, Xin Cong, Xu Han, Yukun Yan, Zhenghao Liu, Zhixing Tan, Pengyuan Liu, Dong Yu, Zhiyuan Liu, Xiaodong Shi, Maosong Sun
| Challenge: | Scientific data visualization is an essential process in research, but its use of large language models remains unexplored. |
| Approach: | They propose a model-agnostic LLM agent framework to automate scientific data visualization tasks. |
| Outcome: | The proposed framework improves performance of commercial and open-source models. |
Similar Papers
DataSciBench: An LLM Agent Benchmark for Data Science (2026.findings-acl)
Copied to clipboard
Dan Zhang, Sining Zhoubian, Min Cai, Fengzu Li, Lekang Yang, Wei Wang, Tianjiao Dong, Ziniu Hu, Jie Tang, Yisong Yue
| Challenge: | Existing benchmarks focus on single task, simple evaluation metrics, and readily available ground truth (GT) DataSciBench is built on curated, natural, and challenging prompts with complex evaluation criteria and uncertain GT. |
| Approach: | They propose a benchmark for evaluating Large Language Models in data science that integrates LLM-based self-consistency and human verification to ensure accuracy. |
| Outcome: | The proposed framework outperforms open-source models in all metrics and offers rigorous insights into LLM strengths and weaknesses. |
LLM-Powered Benchmark Factory: Reliable, Generic, and Efficient (2026.acl-long)
Copied to clipboard
Peiwen Yuan, Shaoxiong Feng, Yiwei Li, Xinglin Wang, Yueqi Zhang, Jiayi Shi, Chuyi Tan, Boyuan Pan, Yao Hu, Kan Li
| Challenge: | Using generic and efficient benchmark generators, human annotators are limited by inefficiency . current benchmark generator methods rely on seed signals, leading to long cycles and high costs . |
| Approach: | They propose a framework to evaluate LLMs as generic benchmark generators and integrate them as BenchMaker. |
| Outcome: | The proposed framework achieves comparable performance to human-annotated benchmarks on most metrics. |
Benchmarking Data Science Agents (2024.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have emerged as promising data science aids, assisting humans in data analysis and processing. |
| Approach: | They propose an evaluation paradigm and benchmarks that assess the performance of data science agents throughout the entire data science lifecycle. |
| Outcome: | The proposed evaluation paradigm streamlines dataset preparation, improves coverage, and expands benchmarking comprehensiveness. |
Large Language Models for Data Annotation and Synthesis: A Survey (2024.emnlp-main)
Copied to clipboard
Zhen Tan, Dawei Li, Song Wang, Alimohammad Beigi, Bohan Jiang, Amrita Bhattacharjee, Mansooreh Karami, Jundong Li, Lu Cheng, Huan Liu
| Challenge: | Existing surveys focus on LLMs' specific utility for data annotation and synthesis. |
| Approach: | They propose to use large language models to generate annotations from raw data . they also propose to review learning strategies for models utilizing LLM-generated annotations . |
| Outcome: | The proposed models can be used to improve the efficacy of machine learning models by generating and labeling raw data with relevant information. |
Agent for Numerical Data Retrieval and Understanding by Code Generation and Multimodal Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Numerical data from sensors and time series are widely used in scientific research fields such as nuclear fusion . efficient numerical data analysis tools are crucial to accelerate experimental research . |
| Approach: | They propose a model-agnostic and data-adic agent that processes numerical data by code generation and multimodal reasoning. |
| Outcome: | The proposed agent outperforms existing methods on benchmarks on sensor data classification and time series understanding. |
LLM4Cell: Taxonomy and Evaluation of LLM and Agentic Models for Single-Cell Biology (2026.acl-long)
Copied to clipboard
Sajib Acharjee Dip, Adrika Zafor, Bikash Kumar Paul, Uddip Acharjee Shuvo, Muhit Islam Emon, Xuan Wang, Liqing Zhang
| Challenge: | Large language models are transforming biomedical discovery by linking molecular patterns with knowledge encoded in text. |
| Approach: | They propose to map 58 foundation and agentic models developed for single-cell research into eight key analytical tasks. |
| Outcome: | The proposed models are applied to eight key analytical tasks including annotation, trajectory inference, perturbation modeling, and drug-response prediction. |
Improving Automatic Evaluation of Large Language Models (LLMs) in Biomedical Relation Extraction via LLMs-as-the-Judge (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models generate human-like text, making them unreliable for biomedical relation extraction tasks. |
| Approach: | They propose to use Large Language Models as judges to evaluate biomedical relation extraction . they propose structured output formatting for LLM-generated responses that helps LLMs improve their performance by 15%. |
| Outcome: | The proposed method improves LLM-Judges' performance by 15% . it is cheaper and more efficient than human evaluation metrics, the authors say . |
Data Interpreter: An LLM Agent for Data Science (2025.findings-acl)
Copied to clipboard
Sirui Hong, Yizhang Lin, Bang Liu, Bangbang Liu, Binhao Wu, Ceyao Zhang, Danyang Li, Jiaqi Chen, Jiayi Zhang, Jinlin Wang, Li Zhang, Lingyao Zhang, Min Yang, Mingchen Zhuge, Taicheng Guo, Tuo Zhou, Wei Tao, Robert Tang, Xiangtao Lu, Xiawu Zheng, Xinbing Liang, Yaying Fei, Yuheng Cheng, Yongxin Ni, Zhibin Gou, Zongze Xu, Yuyu Luo, Chenglin Wu
| Challenge: | Large Language Models (LLMs) excel in various domains but face challenges when applied to data science workflows due to their complex, multi-stage nature. |
| Approach: | They propose a hierarchical graph-based agent that represents complexity and a progressive strategy for step-by-step verification, refinement, and consistent context management. |
| Outcome: | The proposed agent surpasses state-of-the-art baselines on the MATH dataset and performs better on InfiAgent-DABench. |
Text2Vis: A Challenging and Diverse Benchmark for Generating Multimodal Visualizations from Text (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have shown promise in generating visualizations from natural language, but lack of comprehensive benchmarks limits their capabilities. |
| Approach: | They propose a framework that jointly refines the textual answer and visualization code to improve GPT-4o's pass rate from 26% to 42% over direct approach. |
| Outcome: | The proposed framework increases GPT-4o’s pass rate from 26% to 42% over the direct approach and improves chart quality. |
MEGAnno+: A Human-LLM Collaborative Annotation System (2024.eacl-demo)
Copied to clipboard
| Challenge: | Large language models (LLMs) can label data faster and cheaper than humans . however, they may fall short in understanding of complex contexts, leading to incorrect labels . |
| Approach: | They propose a collaborative approach where humans and LLMs work together to produce reliable labels. |
| Outcome: | The proposed system can produce reliable and high-quality labels faster and cheaper than humans . compared to traditional models, it can generate labels faster, at a lower cost . |