Improving Conversational Question Answering Systems after Deployment using Feedback-Weighted Learning (2020.coling-main)
Copied to clipboard
| Challenge: | In most applications, users are not able to provide the correct answer to the system, but they are able provide binary (correct, incorrect) feedback. |
| Approach: | They propose feedback-weighted learning based on importance sampling to improve upon an initial supervised system using binary user feedback. |
| Outcome: | The proposed method improves on an initial supervised system, getting close to a fully-supervised system that has access to the same labeled examples in in-domain experiments (QuAC) and matching in out-of-domain experiment (DoQA). |
Similar Papers
Learning New Skills after Deployment: Improving open-domain internet-driven dialogue with human feedback (2023.acl-long)
Copied to clipboard
| Challenge: | Frozen models trained to mimic static datasets can never improve their performance. |
| Approach: | They propose to use binary quality measurements and free-form text feedback to improve conversational skills in a conversational learning framework. |
| Outcome: | The proposed model improves on the DIRECTOR model, which is based on binary quality measurements and free-form text feedback, and shows that iterative retraining and redeployment can improve the model. |
Using Interactive Feedback to Improve the Accuracy and Explainability of Question Answering Systems Post-Deployment (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing work on question answering focuses on the pre-deployment stage; building an accurate model for deployment. |
| Approach: | They collect feedback from users and train a neural model with the feedback data. |
| Outcome: | The proposed model can explain the correctness or incorrectness of an answer. |
Reinforced Question Rewriting for Conversational Question Answering (2022.emnlp-industry)
Copied to clipboard
| Challenge: | Existing approaches to CQA involve training new models from scratch . existing approaches are expensive and often not feasible . |
| Approach: | They propose to use QA feedback to supervise the rewriting model with reinforcement learning. |
| Outcome: | The proposed model can improve QA performance over baselines for extractive and retrieval QA. |
Ditch the Gold Standard: Re-evaluating Conversational Question Answering (2022.acl-long)
Copied to clipboard
| Challenge: | Existing conversational question answering systems provide natural-language answers to users in information-seeking conversations. |
| Approach: | They conduct the first large-scale human evaluation of state-of-the-art conversational question answering systems . they propose a question rewriting mechanism based on predicted history which better correlates with human judgments . |
| Outcome: | The proposed question rewriting mechanism better correlates with human judgments. |
Simulating Bandit Learning from User Feedback for Extractive Question Answering (2022.acl-long)
Copied to clipboard
| Challenge: | Explicit feedback from users can be used to continually improve system performance. |
| Approach: | They study the potential of learning from user feedback for extractive question answering by simulating feedback using supervised data. |
| Outcome: | The proposed model improves on a few examples and can be deployed in new domains without any data annotation effort. |
ChatR1: Reinforcement Learning for Conversational Reasoning and Retrieval Augmented Question Answering (2026.acl-long)
Copied to clipboard
| Challenge: | Unlike static ‘rewrite, retrieve, and generate’ pipelines, ChatR1 interleaves search and reasoning across turns, enabling exploratory and adaptive behaviors learned through RL. |
| Approach: | They propose a reasoning framework based on reinforcement learning (RL) for conversational question answering that interleaves search and reasoning across turns and provides turn-level feedback. |
| Outcome: | The proposed framework outperforms competing models on five CQA datasets, measured by different metrics (F1, BERTScore, and LLM-as-judge). |
Answer-Supervised Question Reformulation for Enhancing Conversational Machine Comprehension (D19-58)
Copied to clipboard
| Challenge: | Existing question reformulation models are based on supervised question labels without considering feedback information from answers. |
| Approach: | They propose a question reformulation model that integrates conversational history information with reinforcement learning. |
| Outcome: | The proposed model is more effective in conversational machine comprehension with reinforcement learning. |
Leveraging Implicit Feedback from Deployment Data in Dialogue (2024.eacl-short)
Copied to clipboard
| Challenge: | Xu et al., 2023) and Bai ed., 2019) use crowdworkers to collect signals from natural dialogue episodes. |
| Approach: | They use the publicly released BlenderBot deployment data to extract signals from conversations to implicitly measure the quality of a machine-generated utterance. |
| Outcome: | The proposed model improves over baseline models, but some proxy signals can lead to undesirable generations. |
Accurate Training of Web-based Question Answering Systems with Feedback from Ranked Users (2023.acl-industry)
Copied to clipboard
| Challenge: | Recent work shows that large-scale annotated datasets are essential for training state-of-the-art Question Answering (QA) models. |
| Approach: | They use large-scale annotated datasets to train question answering models . they use feedback data collected from deployed QA systems to provide cheaper supervision . |
| Outcome: | The proposed model improves on the large scale annotated datasets from QA systems . the proposed model can be easily supervised on large-scale unlabeled web data . |
RAC: Retrieval-augmented Conversation Dataset for Open-domain Question Answering in Conversational Settings (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Existing studies constrain questions and answers within predefined contexts, excluding the retrieval process. |
| Approach: | They present a retrieval-augmented conversation dataset that addresses key challenges . they propose a system that combines query rewriting and retrieval with reranking . |
| Outcome: | The proposed system improves query rewriting, retrieval, reranking, and response generation performance. |