Improving the Robustness of QA Models to Challenge Sets with Variational Question-Answer Pair Generation (2021.acl-srw)
Copied to clipboard
| Challenge: | Existing data augmentation methods for reading comprehension lack robustness to challenge sets whose distribution is different from that of training sets. |
| Approach: | They propose a question-answer pair generation method that generates multiple diverse QA pairs from a paragraph to mitigate this problem. |
| Outcome: | The proposed model improves the accuracy of 12 challenge sets and the in-distribution accuracy. |
Similar Papers
Improving the Robustness of Question Answering Systems to Question Paraphrasing (P19-1)
Copied to clipboard
| Challenge: | Despite advancement of question answering systems, generalizability of QA models is a topic of concern. |
| Approach: | They propose to use a neural paraphrasing model to generate multiple paraphrased questions for a given source question and a set of paraphrase suggestions to re-train the models. |
| Outcome: | The proposed approach requires no human intervention to re-train the models for improved robustness to question paraphrasing. |
Towards Diverse and Effective Question-Answer Pair Generation from Children Storybooks (2023.findings-acl)
Copied to clipboard
Sugyeong Eo, Hyeonseok Moon, Jinsung Kim, Yuna Hur, Jeongwook Kim, SongEun Lee, Changwoo Chun, Sungsoo Park, Heuiseok Lim
| Challenge: | Recent advances in QA pair generation (QAG) have raised interest in applying this technique to the educational field. |
| Approach: | They propose a QAG framework that enhances QA type diversity by producing different interrogative sentences and implicit/explicit answers. |
| Outcome: | The proposed framework outperforms state-of-the-art methods by significant margins, achieving improved diversity and quality. |
An Empirical Comparison of LM-based Question and Answer Generation Methods (2023.findings-acl)
Copied to clipboard
| Challenge: | Question and answer generation (QAG) is a task of generating question-answer pairs given a context. |
| Approach: | They propose to leverage sequence-to-sequence language model fine-tuning to generate question-answer pairs given a context. |
| Outcome: | The proposed model outperforms other more convoluted approaches in the end-to-end model and is computationally light at both training and inference times. |
Generating Self-Contained and Summary-Centric Question Answer Pairs via Differentiable Reward Imitation Learning (2021.emnlp-main)
Copied to clipboard
| Challenge: | a proposed model for question-answer pairs with self-contained, summary-centric questions and length-constrained, article-summarizing answers is based on suggested question generation in conversational news recommendation systems. |
| Approach: | They propose a model for generating question-answer pairs with self-contained, summary-centric questions and length-constrained, article-summarizing answers. |
| Outcome: | The proposed model captures the central gists of the articles and achieves high answer accuracy. |
Explicit over Implict: Explicit Diversity Conditions for Effective Question Answer Generation (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent pretrained and large language model-based QAG methods suffer from redundant generation of QA pairs, affecting downstream QA systems. |
| Approach: | They propose to use explicit diversity conditions to generate diverse question-answer synthetic data by focusing on spatial aspects, question types, and entities. |
| Outcome: | The proposed diversity conditions significantly increase diversity in QA generation over existing diversity techniques. |
Exploring The Landscape of Distributional Robustness for Question Answering Models (2022.findings-emnlp)
Copied to clipboard
Anas Awadalla, Mitchell Wortsman, Gabriel Ilharco, Sewon Min, Ian Magnusson, Hannaneh Hajishirzi, Ludwig Schmidt
| Challenge: | Existing methods for predicting distributional robustness fail to generalize reliably in a variety of test conditions. |
| Approach: | They conduct a large empirical evaluation to investigate the landscape of distributional robustness in question answering. |
| Outcome: | The proposed methods are more robust to distribution shifts than fully fine-tuned models, and few-shot prompt models exhibit better robustness than few- shot prompt models. |
Handling Anomalies of Synthetic Questions in Unsupervised Question Answering (2020.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to improve unsupervised Question Answering (UQA) are expensive and require additional datasets. |
| Approach: | They propose an unsupervised QA approach that generates QA training data automatically. |
| Outcome: | The proposed method improves unsupervised QA significantly across a number of QA tasks. |
A Practical Toolkit for Multilingual Question and Answer Generation (2023.acl-demo)
Copied to clipboard
| Challenge: | Generating questions and answers from text is a challenging task due to the expected structured output. |
| Approach: | They propose an online service for multilingual QAG along with a python package for model fine-tuning, generation, and evaluation. |
| Outcome: | The proposed model is available in eight languages and can be used online or locally via lmqg. |
It is AI’s Turn to Ask Humans a Question: Question-Answer Pair Generation for Children’s Story Books (2022.acl-long)
Copied to clipboard
| Challenge: | Existing question answering (QA) techniques are created mainly to answer questions asked by humans, but in educational applications, teachers often need to decide what questions to ask . |
| Approach: | They propose to use a fairytale-themed storybook as input to generate QA pairs that can test a student's comprehension skills. |
| Outcome: | The proposed system outperforms state-of-the-art QAG baseline systems and builds an interactive story-telling application for the future real-world deployment. |
Asking Questions Like Educational Experts: Automatically Generating Question-Answer Pairs on Real-World Examination Data (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to generate high quality question-answer pairs are limited . a new framework is proposed for the question-answer generation task on real-world examination data. |
| Approach: | They propose a multi-agent communication model to generate and optimize the question and keyphrases iteratively and then apply the generated question and keys to guide the generation of answers. |
| Outcome: | The proposed framework makes great breakthroughs in the question-answer pair generation task. |