Consistency Training by Synthetic Question Generation for Conversational Question Answering (2024.acl-short)
Copied to clipboard
| Challenge: | Existing methods that use historical information to address user queries in conversational question-answering (CQA) contexts use the gold answers of history instead of the predicted ones. |
| Approach: | They propose a model-agnostic approach that augments historical information with synthetic questions and employs consistency training to implicitly make the reasoning robust to irrelevant history. |
| Outcome: | The proposed model improves in later turns of the conversation when dealing with questions with a large historical context. |
Similar Papers
Conversational QA Dataset Generation with Answer Revision (2022.coling-1)
Copied to clipboard
| Challenge: | Existing frameworks for conversational question-answer generation generate a large-scale dataset based on input passages. |
| Approach: | They propose a conversational question-answer generation framework that extracts question-worthy phrases from passages and generates corresponding questions considering previous conversations. |
| Outcome: | The proposed framework improves the quality of synthetic data and can be used for domain adaptation of conversational question answering. |
Learn to Resolve Conversational Dependency: A Consistency Training Framework for Conversational Question Answering (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches do not explicitly train QA models on how to resolve conversational dependency, and thus these models are limited in understanding human dialogues. |
| Approach: | They propose a framework that generates self-contained questions that can be understood without the conversation history and then trains a QA model with the pairs of original and self-constructed questions using a consistency-based regularizer. |
| Outcome: | The proposed framework improves the models’ performance by up to 1.2 F1 on QuAC, and 5.2 F1 for CANARD, while addressing the limitations of the existing approaches. |
Synthetic QA Corpora Generation with Roundtrip Consistency (P19-1)
Copied to clipboard
| Challenge: | Existing methods for generating synthetic question answering corpora are not suitable for QA, but can be constructed from widely available natural text. |
| Approach: | They propose a method for generating synthetic question answering corpora by combining question generation and answer extraction models and filtering the results to ensure roundtrip consistency. |
| Outcome: | The proposed model achieves exact match and F1 at less than 0.1% and 0.4% from human performance on SQuAD2 and NQ. |
Reinforced Dynamic Reasoning for Conversational Question Generation (P19-1)
Copied to clipboard
| Challenge: | Empirical results on the recently released CoQA dataset demonstrate the effectiveness of our method . large-scale highquality conversational question answering datasets such as CoQA and QuAC can help train models to answer sequential questions. |
| Approach: | They propose a task called Conversational Question Generation which generates a question based on a passage and a conversation history to generate the next question. |
| Outcome: | The proposed method is based on a question-answering style conversation dataset . it can be used to generate meaningful questions on QA and SQuAD datasets . |
Do not let the history haunt you: Mitigating Compounding Errors in Conversational Question Answering (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing approaches employ human-written ground-truth answers for answering conversational questions at test time, but in a realistic scenario, the CoQA model will not have access to ground-Truth answers. |
| Approach: | They propose a sampling strategy that dynamically selects between target answers and model predictions during training, closely simulating the situation at test time. |
| Outcome: | The proposed sampling strategy closely simulates the situation at test time and significantly lowers the performance of CoQA systems. |
Dialogizer: Context-aware Conversational-QA Dataset Generation from Textual Sources (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing dialog inpainting methods generate ConvQA datasets with low contextual relevance due to insufficient learning of question-answer alignment. |
| Approach: | They propose a dialog inpainting method that generates ConvQA datasets from documents . they propose re-ranking tasks and a framework that generate contextually relevant questions . |
| Outcome: | The proposed framework generates ConvQA datasets with high contextual relevance from textual sources. |
Q2: Evaluating Factual Consistency in Knowledge-Grounded Dialogues via Question Generation and Question Answering (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing evaluation methods for factual consistency in knowledge-grounded dialogues are unreliable and limit their applicability. |
| Approach: | They propose an automatic evaluation metric for factual consistency in knowledge-grounded dialogue using automatic question generation and question answering. |
| Outcome: | The proposed evaluation metric consistently shows higher correlation with human judgements. |
CoHS-CQG: Context and History Selection for Conversational Question Generation (2022.coling-1)
Copied to clipboard
| Challenge: | Existing studies focus on single-turn question generation, but few studies have studied the challenges of multiturn QG. |
| Approach: | They propose a two-stage conversational question generation framework that shortens the context and history of the input and calculates relevance scores. |
| Outcome: | The proposed framework achieves state-of-the-art on CoQA in answer-aware and answer-unaware settings. |
Training Question Answering Models From Synthetic Data (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on question and answer generation aims to improve question answering models given limited amount of labeled data. |
| Approach: | They synthesize questions and answers from a synthetic text corpus generated by an 8.3 billion parameter GPT-2 model and achieve 88.4 Exact Match (EM) and 93.9 F1 score on the SQuAD1.1 dev set. |
| Outcome: | The proposed model achieves higher accuracy than the SQUAD1.1 training set questions using synthetic questions and answers than the training set question. |
ProtoQA: A Question Answering Dataset for Prototypical Common-Sense Reasoning (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing question answering datasets for common sense reasoning are lacking for prototypical situations. |
| Approach: | They propose a question answering dataset for training and evaluating common sense reasoning capabilities of artificial intelligence systems in such prototypical situations. |
| Outcome: | The proposed model outperforms existing models on all evaluation metrics with a meaningful gap. |