SAGE: Steerable Agentic Data Generation for Deep Search with Execution Feedback (2026.findings-eacl)
Copied to clipboard
Fangyuan Xu, Rujun Han, Yanfei Chen, Zifeng Wang, I-Hung Hsu, Jun Yan, Vishy Tirumalashetty, Eunsol Choi, Tomas Pfister, Chen-Yu Lee
| Challenge: | High-quality, complex question-answer pairs are pivotal for training and evaluating capable deep search agents. |
| Approach: | They propose a pipeline that generates high-quality, difficulty-controlled deep search question-answer pairs for a given corpus and a target difficulty level. |
| Outcome: | The proposed pipeline generates high-quality, difficulty-controlled deep search question-answer pairs for a given corpus and a target difficulty level. |
Similar Papers
Synthesizing question answering data from financial documents: An End-to-End Multi-Agent Approach (2026.eacl-industry)
Copied to clipboard
| Challenge: | Large language models excel at financial reasoning but their deployment for enterprise use cases remains costly and often constrained by latency, privacy, and regulatory requirements. |
| Approach: | They propose a pipeline that extracts and selects relevant content from unstructured financial documents and generates QA pairs from the selected content for SLM fine-tuning. |
| Outcome: | The proposed model outperforms models trained on previous manual models and achieves competitive in-distribution performance. |
AutoSearch: Adaptive Search Depth for Efficient Agentic RAG via Reinforcement Learning (2026.findings-acl)
Copied to clipboard
Jingbo Sun, Wenyue Chong, Songjun Tu, Qichao Zhang, Yaocheng Zhang, Jiajun Chai, Xiaohan Wang, Wei Lin, Guojun Yin, Dongbin Zhao
| Challenge: | Prior work limits search depth to reduce cost, but this often leads to underexploration of complex questions. |
| Approach: | They propose a reinforcement learning framework that evaluates each search step via self-generated intermediate answers. |
| Outcome: | Extensive experiments on multiple benchmarks show that AutoSearch achieves a superior accuracy-efficiency trade-off, alleviating over-searching while preserving search quality. |
A Pipeline for Generating, Annotating and Employing Synthetic Data for Real World Question Answering (2022.emnlp-demos)
Copied to clipboard
| Challenge: | Question Answering (QA) is a growing area of research . state-of-the-art QA models struggle on out-of domain documents without fine-tuning . |
| Approach: | They propose a pipeline for validating and training QA data and an interface for human annotation. |
| Outcome: | The proposed pipeline improves QA performance on domain-specific datasets while preserving the accuracy of the model. |
SAGE: A Search-AuGmented Evaluation of Large Language Models on Free-Form QA (2026.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are prone to hallucination and rely on static, pre-annotated references for evaluation. |
| Approach: | They propose a framework to assess large language models without fixed ground-truth answers by iteratively generating web queries and synthesizing external evidence. |
| Outcome: | The proposed framework achieves substantial to perfect agreement with human evaluations on multiple free-form QA benchmarks. |
TARGA: Targeted Synthetic Data Generation for Practical Reasoning over Structured Data (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for semantic parsing rely on extensive manually annotated datasets and limited generalization capability to unseen examples. |
| Approach: | They propose a framework that generates high-relevance synthetic data without manual annotation . they generate queries for the queries and use them as demonstrations for in-context learning . |
| Outcome: | The proposed framework outperforms non-fine-tuned methods on KBQA datasets and shows superior sample efficiency, robustness, and generalization capabilities under non-I.I.D. settings. |
Data-Centric Perspectives on Agentic Retrieval-Augmented Generation: A Survey (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) excel at natural language understanding and generation, yet rely on static pre-training data. |
| Approach: | They propose to augment Large Language Models with external retrieval to ground model outputs . traditional RAG is constrained by a fixed retrieve-then-generate routine . authors aim to guide creation of high-quality datasets for next generation of adaptive LLM agents . |
| Outcome: | The proposed model can decompose tasks, issue exploratory queries, and refine evidence through iterative retrieval. |
SynDARin: Synthesising Datasets for Automated Reasoning in Low-Resource Languages (2025.coling-main)
Copied to clipboard
| Challenge: | Question Answering datasets are scarce for languages other than English due to the cost and difficulties of collection and manual annotation. |
| Approach: | They propose a method for generating and validating QA datasets for low-resource languages . they use English data as context to generate synthetic multiple-choice (MC) question-answer pairs . |
| Outcome: | The proposed method maintains quality, reduces likelihood of factual errors, and circumvents costly annotation. |
Towards Faithful and Robust LLM Specialists for Evidence-Based Question-Answering (2024.acl-long)
Copied to clipboard
| Challenge: | Evidence-Based QA has proved insufficiently faithful with Large Language Models . a typical application of LLMs is in Evidence-based Question Answering (QA). |
| Approach: | They propose a data generation pipeline with automated data quality filters to fine-tune LLMs for better source quality and answer attributability. |
| Outcome: | The proposed model can synthesize high-quality training and testing data at scale. |
SPINACH: SPARQL-Based Information Navigation for Challenging Real-World Questions (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have led to significant improvements in the Knowledge Base Question Answering task. |
| Approach: | They introduce an expert-annotated KBQA dataset from Wikidata’s “Request a Query” forum with 320 decontextualized question-SPARQL pairs. |
| Outcome: | The SPINACH dataset outperforms baselines on the QALD-7, QADL-9 Plus and QAL-10 datasets by 31.0%, 27.0% and 10.0% in F1 respectively. |
SATQuest: A Verifier for Logical Reasoning Evaluation and Reinforcement Fine-Tuning of LLMs (2026.acl-long)
Copied to clipboard
Yanxiao Zhao, Yaqian Li, Zi-Hao Bo, Rinyoichi Takezoe, Haojia Hui, Mo Guang, null Renlei, Xiaolin Qin, Kaiwen Long
| Challenge: | Large language models exhibit strong general reasoning abilities, yet the community lacks controllable, scalable, and verifiable tools to analyze and improve them. |
| Approach: | They propose a verifier that generates diverse SAT-based reasoning tasks from CNF instances and checks answers objectively with PySAT. |
| Outcome: | The proposed verifier generates diverse SAT-based reasoning tasks from CNF instances and checks answers objectively with PySAT. |