Learning to Ask Unanswerable Questions for Machine Reading Comprehension (P19-1)
Copied to clipboard
| Challenge: | Existing models for extractive reading comprehension are not good at deciding whether no answer is presented in the context. |
| Approach: | They propose a data augmentation technique by automatically generating relevant unanswerable questions according to an answerable question paired with its corresponding paragraph that contains the answer. |
| Outcome: | The proposed model performs better on the SQuAD 2.0 dataset than the baseline model and the BERT-large model. |
Similar Papers
Know What You Don’t Know: Unanswerable Questions for SQuAD (P18-2)
Copied to clipboard
| Challenge: | Existing datasets focus on answerable questions or use automatically generated unanswerable questions that are easy to identify. |
| Approach: | They propose a dataset that combines the Stanford Question Answering Dataset with 50,000 unanswerable questions written by crowdworkers to look similar to answerable ones. |
| Outcome: | The proposed dataset looks similar to answerable questions on crowd-written questions . strong neural system that gets 86% F1 on SQuAD achieves only 66% F1. |
SQuAD2-CR: Semi-supervised Annotation for Cause and Rationales for Unanswerability in SQuAD 2.0 (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing models are brittle for adversarial perturbed questions, causing uncertainty . a dataset with annotations on unanswerable questions is not available to solve this problem . |
| Approach: | They use crowdsourced annotations to annotate unanswerable questions . they also annotated which part of the question causes unanswered questions a . |
| Outcome: | The proposed dataset can be used to improve model interpretation, authors say . they find that existing models are brittle for adversarial perturbed questions . |
A Lightweight Method to Generate Unanswerable Questions in English (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to build robust question answering models are too complex . antonym and entity swaps on answerable questions are used to build models . |
| Approach: | They propose a method for performing antonym and entity swaps on unanswerable questions. |
| Outcome: | The proposed method outperforms the previous state-of-the-art and has higher human-judged relatedness and readability. |
Training Question Answering Models From Synthetic Data (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on question and answer generation aims to improve question answering models given limited amount of labeled data. |
| Approach: | They synthesize questions and answers from a synthetic text corpus generated by an 8.3 billion parameter GPT-2 model and achieve 88.4 Exact Match (EM) and 93.9 F1 score on the SQuAD1.1 dev set. |
| Outcome: | The proposed model achieves higher accuracy than the SQUAD1.1 training set questions using synthetic questions and answers than the training set question. |
The Impacts of Unanswerable Questions on the Robustness of Machine Reading Comprehension Models (2023.eacl-main)
Copied to clipboard
| Challenge: | Pretrained language models have achieved super-human performances on many Machine Reading Comprehension (MRC) benchmarks. |
| Approach: | They propose to fine-tune three state-of-the-art language models on SQuAD 1.1 or SQu AD 2.0 and then evaluate their robustness under adversarial attacks. |
| Outcome: | The proposed model is able to perform better under adversarial attacks than model fine-tuned on SQuAD 1.1 or 2.0. |
Unsupervised FAQ Retrieval with Question Generation and BERT (2020.acl-main)
Copied to clipboard
| Challenge: | Frequently Asked Questions (FAQ) retrieval requires labeled datasets for training neural models. |
| Approach: | They propose to exploit FAQ pairs to train two BERT models that match user queries to FAQ answers and questions. |
| Outcome: | The proposed model outperforms supervised models on existing datasets and is on par with existing dataset. |
A Recurrent BERT-based Model for Question Generation (D19-58)
Copied to clipboard
| Challenge: | Existing QG models rely on recurrent neural networks (RNNs) but the inherent sequential nature of the RNN models suffers from the problem of handling long sequences. |
| Approach: | They propose to employ a pre-trained BERT language model to tackle question generation tasks. |
| Outcome: | The proposed model outperforms the existing models on the question-answering dataset SQuAD and advances the BLEU 4 score from 16.85 to 22.17. |
Learning to Generate Questions by Learning to Recover Answer-containing Sentences (2021.findings-acl)
Copied to clipboard
| Challenge: | Recent research has focused on synthetically generating a question from a given context and an annotated answer by training an additional generative model. |
| Approach: | They propose a method that learns to generate contextually rich questions by recovering answer-containing sentences. |
| Outcome: | The proposed approach improves the quality and accuracy of existing models and achieves comparable results to the state-of-the-art on MS MARCO and NewsQA. |
Simple and Effective Multi-Paragraph Reading Comprehension (P18-1)
Copied to clipboard
| Challenge: | Existing question answering models cannot scale beyond short paragraphs, so adapting a model to document-level input is difficult. |
| Approach: | They propose a method of adapting neural paragraph-level question answering models to document input. |
| Outcome: | The proposed method achieves state-of-the-art on TriviaQA and SQuAD and a 10 point gain on SQuADA. |
MRC Examples Answerable by BERT without a Question Are Less Effective in MRC Model Training (2020.aacl-srw)
Copied to clipboard
| Challenge: | Existing models for Machine Reading Comprehension (MRC) are unable to predict answers from a question and its related context. |
| Approach: | They propose a method that splits the training examples into those that are “easy to answer” or “hard to answer”. |
| Outcome: | The proposed model outperforms the previous models on a large-scale English MRC dataset. |