AutoDCWorkflow: LLM-based Data Cleaning Workflow Auto-Generation and Benchmark (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Data cleaning is a time-consuming and error-prone manual process even with modern workflow tools like OpenRefine. |
| Approach: | AutoDCWorkflow generates a table with a data analysis purpose and generates an open-refine workflow. |
| Outcome: | The proposed pipeline generates clean, minimal tables for data analysis tasks. |
Similar Papers
Beyond Traditional Benchmarks: Analyzing Behaviors of Open LLMs on Data-to-Text Generation (2024.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks for data-to-text generation are saturated, and there is no way to test them. |
| Approach: | They propose a tool for collecting structured data from public APIs to analyze the behavior of open large language models on the task of data-to-text generation. |
| Outcome: | The proposed model can generate fluent and coherent texts in zero-shot settings from data in common formats collected with Quintd. |
ComfyFlow: Benchmarking LLMs for AIGC Workflow Generation (2026.findings-acl)
Copied to clipboard
Zhenran Xu, Yiyu Wang, Yunxin li, Muyang Ye, null Yangxue, Kai Chen, Longyue Wang, Weihua Luo, Baotian Hu, Min Zhang
| Challenge: | Large language models (LLMs) have shown promising advances in tackling human-level tasks, but generating workflows for collaborative AI systems remains a critical and challenging step. |
| Approach: | They propose a benchmark to evaluate LLMs’ ability to generate executable and instruction-following AIGC workflows in ComfyUI. |
| Outcome: | The proposed benchmarks show that LLMs can generate executable and instruction-following AIGC workflows in ComfyUI. |
AutoSDT: Scaling Data-Driven Discovery Tasks Toward Open Co-Scientists (2025.emnlp-main)
Copied to clipboard
Yifei Li, Hanane Nour Moussa, Ziru Chen, Shijie Chen, Botao Yu, Mingyi Xue, Benjamin Burns, Tzu-Yao Chiu, Vishal Dey, Zitong Lu, Chen Wei, Qianheng Zhang, Tianyu Zhang, Song Gao, Xuhui Huang, Xia Ning, Nesreen K. Ahmed, Ali Payani, Huan Sun
| Challenge: | AutoSDT-5K is the only automatically collected and the largest open dataset for data-driven scientific discovery. |
| Approach: | They propose an automatic pipeline that collects high-quality coding tasks in real-world data-driven discovery workflows. |
| Outcome: | The proposed pipeline synthesizes accurate tasks and tasks from a dataset of 5,404 tasks covering four scientific disciplines and 756 Python packages. |
AIRepr: An Analyst-Inspector Framework for Evaluating Reproducibility of LLMs in Data Science (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models are increasingly used to automate data analysis, but data science tasks often admit multiple statistically valid solutions. |
| Approach: | They propose a framework to evaluate LLM-generated code and assess its reproducibility . they introduce two reproducibility-enhancing prompting strategies and benchmark them against standard prompting . |
| Outcome: | The proposed framework improves reproducibility of large language models . it provides a foundation for transparent, reliable, and efficient human–AI collaboration in data science. |
CiteLab: Developing and Diagnosing LLM Citation Generation Workflows via the Human-LLM Interaction (2025.acl-demo)
Copied to clipboard
| Challenge: | Existing frameworks for enabling Large Language Models to generate citations are lacking . however, they can still produce hallucinated responses that are non-factual or irrelevant to the input. |
| Approach: | They propose an open-source and modular framework for enabling LLMs to generate citations in Question-Answering tasks. |
| Outcome: | The proposed framework is extensible and paired with a visual interface, Citefix, facilitating case study and modification of existing citation generation methods. |
AntiLeakBench: Preventing Data Contamination by Automatically Constructing Benchmarks with Updated Real-World Knowledge (2025.acl-long)
Copied to clipboard
Xiaobao Wu, Liangming Pan, Yuxi Xie, Ruiwen Zhou, Shuai Zhao, Yubo Ma, Mingzhe Du, Rui Mao, Anh Tuan Luu, William Yang Wang
| Challenge: | Existing studies solve this challenge by updating benchmarks with newly collected data, but they fail to guarantee contamination-free evaluation as the newly collected knowledge may contain pre-existing knowledge. |
| Approach: | They propose an automated anti-leakage benchmarking framework that builds and updates benchmarks without human labor instead of using newly collected data. |
| Outcome: | The proposed framework significantly reduces the cost of benchmark maintenance to accommodate emerging LLMs. |
C2: Scalable Auto-Feedback for LLM-based Chart Generation (2025.naacl-long)
Copied to clipboard
Woosung Koh, Janghan Yoon, MinHyung Lee, Youngjin Song, Jaegwan Cho, Jaehyun Kang, Taehyeon Kim, Se-Young Yun, Youngjae Yu, Bongshin Lee
| Challenge: | generating high-quality charts with Large Language Models presents significant challenges due to limited data and the high cost of curation. |
| Approach: | They propose a referencefree automatic feedback generator to generate high-quality charts with Large Language Models. |
| Outcome: | The proposed framework outperforms baselines and shows that it significantly improves data diversity. |
Bridging the Editing Gap in LLMs: FineEdit for Precise and Targeted Text Modifications (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study shows that large language models can perform precise text editing tasks. |
| Approach: | InstrEditBench is a benchmark dataset that compares 30,000 structured editing tasks . experimental evaluations show FineEdit outperforms state-of-the-art models . |
| Outcome: | The proposed model outperforms state-of-the-art models on single-turn edits and mistral-7B-OpenOrca on direct edits. |
LLM-Based Web Data Collection for Research Dataset Creation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | researchers across many fields rely on web data to gain new insights and validate methods. |
| Approach: | They propose a human-in-the-loop framework that automates web-scale data collection end-to-end using large language models (LLMs) |
| Outcome: | The proposed framework outperforms existing methods in three different tasks and a user evaluation demonstrates its practical utility. |
On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Survey (2024.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) provide a data-centric solution to alleviate limitations of real-world data with synthetic data generation. |
| Approach: | They propose a generic workflow for LLM-driven synthetic data generation. |
| Outcome: | The proposed workflows highlight gaps in existing research and outline avenues for future studies. |