LLM-Based Web Data Collection for Research Dataset Creation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | researchers across many fields rely on web data to gain new insights and validate methods. |
| Approach: | They propose a human-in-the-loop framework that automates web-scale data collection end-to-end using large language models (LLMs) |
| Outcome: | The proposed framework outperforms existing methods in three different tasks and a user evaluation demonstrates its practical utility. |
Similar Papers
On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Survey (2024.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) provide a data-centric solution to alleviate limitations of real-world data with synthetic data generation. |
| Approach: | They propose a generic workflow for LLM-driven synthetic data generation. |
| Outcome: | The proposed workflows highlight gaps in existing research and outline avenues for future studies. |
The Data Frontier for Large Language Models: Selection, Synthesis, and Tools (2026.acl-tutorials)
Copied to clipboard
| Challenge: | acquiring and curating high-quality training data remains a significant bottleneck . acquiring such high-quality data is a key challenge for researchers and practitioners . |
| Approach: | This tutorial provides a comprehensive and practical guide to the state-of-the-art in data research directions for LLMs. |
| Outcome: | The tutorial covers methods for curating the most valuable information from vast, noisy datasets and the synthetic data revolution. |
Large Language Models are Built-in Autoregressive Search Engines (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing dual-encoder dense retrievers obtain representations for questions and documents independently, allowing only shallow interactions between them. |
| Approach: | They propose to use large language models to generate URLs for document retrieval by following human instructions. |
| Outcome: | The proposed method achieves better retrieval performance than existing retrieval approaches on open-domain question answering benchmarks. |
DataDreamer: A Tool for Synthetic Data Generation and Reproducible LLM Workflows (2024.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have become a dominant tool for NLP researchers in a wide range of tasks. |
| Approach: | They propose an open source Python library that allows researchers to write simple code to implement powerful LLM workflows. |
| Outcome: | The proposed library is open source and can be used to implement powerful LLM workflows. |
An LLM-Based Approach for Insight Generation in Data Analysis (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing approaches to generate insightful data from databases are time-consuming and resource-intensive. |
| Approach: | They propose a method that leverages Large Language Models to automatically generate textual insights from databases. |
| Outcome: | The proposed approach generates more insightful insights than other approaches while maintaining correctness. |
InsightPilot: An LLM-Empowered Automated Data Exploration System (2023.emnlp-demo)
Copied to clipboard
| Challenge: | InsightPilot is an LLM-based, automated data exploration system designed to simplify the data exploration process. |
| Approach: | They propose an LLM-based, automated data exploration system that streamlines the data exploration process. |
| Outcome: | InsightPilot is an LLM-based, automated data exploration system . it can help users gain valuable insights from their datasets, in a case study and in nl . |
TinyScientist: An Interactive, Extensible, and Controllable Framework for Building Research Agents (2025.emnlp-demos)
Copied to clipboard
Haofei Yu, Keyang Xuan, Fenghai Li, Kunlun Zhu, Zijie Lei, Jiaxun Zhang, Ziheng Qi, Kyle Richardson, Jiaxuan You
| Challenge: | Existing research systems often design and use agentic workflows to perform research tasks such as ideation, scientific coding, review writing, and tree-based search. |
| Approach: | They propose an open-source codebase, an interactive web demonstration, and a PyPI Python package to make state-of-the-art auto-research pipelines broadly accessible to every researcher and developer. |
| Outcome: | The proposed framework adapts easily to new tools and supports iterative growth. |
LLM-in-the-loop: Leveraging Large Language Model for Thematic Analysis (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent research shows that large language models can replicate human-like behavior in various tasks. |
| Approach: | They propose a framework for human-LLM collaboration to conduct TA with in-context learning (ICL) they propose to use survey data to frame discussions with an LLM to generate a final codebook for TA. |
| Outcome: | The proposed framework outperforms crowd workers on text-annotation tasks and yields similar coding quality to that of human coders but reduces TA’s labor and time demands. |
ResearchArena: Benchmarking Large Language Models’ Ability to Collect and Organize Information as Research Agents (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models excel across many natural language processing tasks but face challenges in domain-specific, analytical tasks such as conducting research surveys. |
| Approach: | They propose a benchmark to evaluate LLMs' capabilities in conducting research surveys. |
| Outcome: | The proposed benchmark is designed to evaluate LLMs' capabilities in conducting research surveys. |
Easy Dataset: A Unified and Extensible Framework for Synthesizing LLM Fine-Tuning Data from Unstructured Documents (2025.emnlp-demos)
Copied to clipboard
| Challenge: | Existing data synthesis tools struggle to extract reliable fine-tuning data from heterogeneous documents. |
| Approach: | They propose a framework for synthesizing fine-tuning data from unstructured documents via an intuitive graphical user interface. |
| Outcome: | The proposed framework can extract reliable data from unstructured documents via an intuitive graphical user interface (GUI) it leverages persona-driven prompting approach to generate diverse question-answer pairs using public-available LLMs. |