A Pipeline for Generating, Annotating and Employing Synthetic Data for Real World Question Answering (2022.emnlp-demos)
Copied to clipboard
| Challenge: | Question Answering (QA) is a growing area of research . state-of-the-art QA models struggle on out-of domain documents without fine-tuning . |
| Approach: | They propose a pipeline for validating and training QA data and an interface for human annotation. |
| Outcome: | The proposed pipeline improves QA performance on domain-specific datasets while preserving the accuracy of the model. |
Similar Papers
End-to-End Synthetic Data Generation for Domain Adaptation of Question Answering Systems (2020.emnlp-main)
Copied to clipboard
Siamak Shakeri, Cicero Nogueira dos Santos, Henghui Zhu, Patrick Ng, Feng Nan, Zhiguo Wang, Ramesh Nallapati, Bing Xiang
| Challenge: | Existing approaches for synthetic QA data generation have limited or no success in improving the downstream Reading Comprehension task. |
| Approach: | They propose an end-to-end approach for synthetic QA data generation using a transformer-based encoder-decoder network that is trained end- to-end to generate both answers and questions. |
| Outcome: | The proposed model outperforms current state-of-the-art methods in the domain adaptation of QA models. |
Synthetic Question Value Estimation for Domain Adaptation of Question Answering (2022.acl-long)
Copied to clipboard
| Challenge: | Existing work adapts QA scores to select high-quality questions, but these scores do not improve QA performance on the target domain. |
| Approach: | They propose to synthesize QA pairs with a question generator on the target domain . they propose to train a Question Value Estimator that estimates usefulness of synthetic questions . |
| Outcome: | The proposed method improves the performance of the target domain QA model by using synthetic questions and only 15% of the human annotations on the targetdomain. |
Synthesizing question answering data from financial documents: An End-to-End Multi-Agent Approach (2026.eacl-industry)
Copied to clipboard
| Challenge: | Large language models excel at financial reasoning but their deployment for enterprise use cases remains costly and often constrained by latency, privacy, and regulatory requirements. |
| Approach: | They propose a pipeline that extracts and selects relevant content from unstructured financial documents and generates QA pairs from the selected content for SLM fine-tuning. |
| Outcome: | The proposed model outperforms models trained on previous manual models and achieves competitive in-distribution performance. |
Leveraging QA Datasets to Improve Generative Data Augmentation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in NLP have substantially improved the capability of pretrained language models to generate high-quality text. |
| Approach: | They propose to reformulate data generation as context generation for a given question-answer (QA) pair and leverage QA datasets for training context generators. |
| Outcome: | The proposed approach improves performance for few-shot and zero-shot classification datasets on multiple classification dataset. |
Handling Anomalies of Synthetic Questions in Unsupervised Question Answering (2020.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to improve unsupervised Question Answering (UQA) are expensive and require additional datasets. |
| Approach: | They propose an unsupervised QA approach that generates QA training data automatically. |
| Outcome: | The proposed method improves unsupervised QA significantly across a number of QA tasks. |
Unsupervised Adaptation of Question Answering Systems via Generative Self-training (2020.emnlp-main)
Copied to clipboard
| Challenge: | Supervised self-training methods have transformed applied machine learning . however, adapting to target data has received little attention . |
| Approach: | They propose a method to generate synthetic QA pairs for unsupervised self adaptation . they use massive amounts of data to simulate self-supervised tasks . |
| Outcome: | The proposed method improves QA systems significantly by using less data and training computation than existing augmentation approaches. |
Synthetic QA Corpora Generation with Roundtrip Consistency (P19-1)
Copied to clipboard
| Challenge: | Existing methods for generating synthetic question answering corpora are not suitable for QA, but can be constructed from widely available natural text. |
| Approach: | They propose a method for generating synthetic question answering corpora by combining question generation and answer extraction models and filtering the results to ensure roundtrip consistency. |
| Outcome: | The proposed model achieves exact match and F1 at less than 0.1% and 0.4% from human performance on SQuAD2 and NQ. |
NoiseQA: Challenge Set Evaluation for User-Centric Question Answering (2021.eacl-main)
Copied to clipboard
| Challenge: | Question-Answering (QA) systems are deployed in the real world . a lack of research attention has been devoted to studying the issues that arise when people use QA systems. |
| Approach: | They show that component components that precede an answering engine can introduce varied and considerable sources of error. |
| Outcome: | The proposed evaluations highlight the need for QA evaluation to expand to consider real-world use. |
Accurate Training of Web-based Question Answering Systems with Feedback from Ranked Users (2023.acl-industry)
Copied to clipboard
| Challenge: | Recent work shows that large-scale annotated datasets are essential for training state-of-the-art Question Answering (QA) models. |
| Approach: | They use large-scale annotated datasets to train question answering models . they use feedback data collected from deployed QA systems to provide cheaper supervision . |
| Outcome: | The proposed model improves on the large scale annotated datasets from QA systems . the proposed model can be easily supervised on large-scale unlabeled web data . |
Towards Multi-Document Question Answering in Scientific Literature: Pipeline, Dataset, and Evaluation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing QA systems do not strictly enforce cross-document synthesis or exploit the explicit inter-paper structure that links sources. |
| Approach: | They propose a pipeline methodology for constructing a multi-document academic QA dataset . they detect communities based on citation networks and leverage Large Language Models . |
| Outcome: | The proposed method generates QA pairs related to multi-document content automatically and forms coherent communities based on citation networks and large language models. |