| Challenge: | Existing methods for generating synthetic question answering corpora are not suitable for QA, but can be constructed from widely available natural text. |
| Approach: | They propose a method for generating synthetic question answering corpora by combining question generation and answer extraction models and filtering the results to ensure roundtrip consistency. |
| Outcome: | The proposed model achieves exact match and F1 at less than 0.1% and 0.4% from human performance on SQuAD2 and NQ. |
Similar Papers
Training Question Answering Models From Synthetic Data (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on question and answer generation aims to improve question answering models given limited amount of labeled data. |
| Approach: | They synthesize questions and answers from a synthetic text corpus generated by an 8.3 billion parameter GPT-2 model and achieve 88.4 Exact Match (EM) and 93.9 F1 score on the SQuAD1.1 dev set. |
| Outcome: | The proposed model achieves higher accuracy than the SQUAD1.1 training set questions using synthetic questions and answers than the training set question. |
Unsupervised Adaptation of Question Answering Systems via Generative Self-training (2020.emnlp-main)
Copied to clipboard
| Challenge: | Supervised self-training methods have transformed applied machine learning . however, adapting to target data has received little attention . |
| Approach: | They propose a method to generate synthetic QA pairs for unsupervised self adaptation . they use massive amounts of data to simulate self-supervised tasks . |
| Outcome: | The proposed method improves QA systems significantly by using less data and training computation than existing augmentation approaches. |
A Pipeline for Generating, Annotating and Employing Synthetic Data for Real World Question Answering (2022.emnlp-demos)
Copied to clipboard
| Challenge: | Question Answering (QA) is a growing area of research . state-of-the-art QA models struggle on out-of domain documents without fine-tuning . |
| Approach: | They propose a pipeline for validating and training QA data and an interface for human annotation. |
| Outcome: | The proposed pipeline improves QA performance on domain-specific datasets while preserving the accuracy of the model. |
PAXQA: Generating Cross-lingual Question Answering Examples at Training Scale (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing question answering systems rely on large, high-quality training data. |
| Approach: | They propose a synthetic data generation method which decomposes cross-lingual QA into two stages . they apply a question generation model to the English side and annotation projection to translate both questions and answers. |
| Outcome: | The proposed method outperforms existing methods on cross-lingual QA datasets. |
Conversational QA Dataset Generation with Answer Revision (2022.coling-1)
Copied to clipboard
| Challenge: | Existing frameworks for conversational question-answer generation generate a large-scale dataset based on input passages. |
| Approach: | They propose a conversational question-answer generation framework that extracts question-worthy phrases from passages and generates corresponding questions considering previous conversations. |
| Outcome: | The proposed framework improves the quality of synthetic data and can be used for domain adaptation of conversational question answering. |
End-to-End Synthetic Data Generation for Domain Adaptation of Question Answering Systems (2020.emnlp-main)
Copied to clipboard
Siamak Shakeri, Cicero Nogueira dos Santos, Henghui Zhu, Patrick Ng, Feng Nan, Zhiguo Wang, Ramesh Nallapati, Bing Xiang
| Challenge: | Existing approaches for synthetic QA data generation have limited or no success in improving the downstream Reading Comprehension task. |
| Approach: | They propose an end-to-end approach for synthetic QA data generation using a transformer-based encoder-decoder network that is trained end- to-end to generate both answers and questions. |
| Outcome: | The proposed model outperforms current state-of-the-art methods in the domain adaptation of QA models. |
Harvesting and Refining Question-Answer Pairs for Unsupervised QA (2020.acl-main)
Copied to clipboard
| Challenge: | Recent research attempts to extend unsupervised question answering to settings with few or no labeled data available. |
| Approach: | They propose two approaches to improve unsupervised question answering . first, they harvest lexically and syntactically divergent Wikipedia questions to automatically construct a corpus of question-answer pairs . second, they take advantage of the QA model to extract more appropriate answers . |
| Outcome: | The proposed approach outperforms previous unsupervised approaches by a large margin and is competitive with early supervised models. |
Handling Anomalies of Synthetic Questions in Unsupervised Question Answering (2020.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to improve unsupervised Question Answering (UQA) are expensive and require additional datasets. |
| Approach: | They propose an unsupervised QA approach that generates QA training data automatically. |
| Outcome: | The proposed method improves unsupervised QA significantly across a number of QA tasks. |
Vocabulary Matters: A Simple yet Effective Approach to Paragraph-level Question Generation (2020.aacl-main)
Copied to clipboard
| Challenge: | Current neural network-based questions generation techniques take only one or two sentences as input. |
| Approach: | They propose a simple yet effective technique for question generation from paragraphs . they augment a sequence-to-sequence QG model with dynamic, paragraph-specific dictionary . |
| Outcome: | The proposed model outperforms state-of-the-art systems in question generation from paragraphs in automatic and human evaluation. |
Fluent Response Generation for Conversational Question Answering (2020.acl-main)
Copied to clipboard
| Challenge: | Question answering (QA) is an important aspect of open-domain conversational agents, garnering specific research focus in the conversational QA subtask. |
| Approach: | They propose a method for situating QA responses within a SEQ2SEQ NLG approach to generate fluent grammatical answer responses while maintaining correctness. |
| Outcome: | The proposed model outperforms baseline CoQA and QuAC models in generating conversational responses. |