Constrained Policy Optimization for Controlled Self-Learning in Conversational AI Systems (2023.acl-industry)
Copied to clipboard
| Challenge: | Recent self-learning methods based on user satisfaction metrics and contextual bandits have shown promising results to enable consistent improvements in conversational AI systems. |
| Approach: | They propose a meta-gradient learning approach that adjusts constraint violation penalty terms adaptively through a user-defined meta objective that encourages balanced constraint satisfaction across domains. |
| Outcome: | The proposed framework supports fine-grained exploration targets for individual domains via user-defined constraints. |
Similar Papers
Scalable and Robust Self-Learning for Skill Routing in Large-Scale Conversational AI Systems (2022.naacl-industry)
Copied to clipboard
| Challenge: | Existing methods to enable skill routing do not scale in terms of the number of skills and skill on-boarding. |
| Approach: | They propose a model-based approach to enable natural conversation by allowing frequent policy updates . they propose an annotation-based system, rule-based model, and bandit-based learning . |
| Outcome: | The proposed method is scalable and cost-effective, the authors show . they show that it can improve the user experience without abrupt policy changes . |
One Planner To Guide Them All ! Learning Adaptive Conversational Planners for Goal-oriented Dialogues (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for goal-oriented dialogues involve training separate models for specific combinations of objectives, leading to computational and scalability issues. |
| Approach: | They propose a new dialogue policy method that can adapt to varying objective preferences at inference time without retraining. |
| Outcome: | The proposed method can adapt to varying objective preferences at inference time without retraining. |
Few-Shot Structured Policy Learning for Multi-Domain and Multi-Task Dialogues (2023.findings-eacl)
Copied to clipboard
| Challenge: | Reinforcement learning is widely adopted to model dialogue managers in task-oriented dialogues, but the user simulator provided by state-of-the-art dialogue frameworks are only rough approximations of human behaviour. |
| Approach: | They propose to use structured policies to improve sample efficiency when learning on multi-domain and multi-task environments. |
| Outcome: | The proposed policies improve sample efficiency and performance on multi-domain and multi-task environments. |
Self-Aware Feedback-Based Self-Learning in Large-Scale Conversational AI (2022.naacl-industry)
Copied to clipboard
| Challenge: | Large-scale conversational AI systems require constant update to adapt to changing customer behavior and trends . lack of self-awareness in feedback-based systems can cause degradation of performance . et al., e. alderman and scott k. d. argues that such systems are not scalable enough to sustain the rapid update pace of conversational systems. |
| Approach: | They propose a superposition-based model that reactively learns local-adaptive decision boundaries . they propose rewritings with a bi-variate beta setting to improve the model's performance . |
| Outcome: | The proposed model improves the PR-AUC by 27.45% and reduces relative defect reductions by 31.22% . the proposed model can adapt faster to changes in global preferences across a large number of customers . |
Guided Dialog Policy Learning: Reward Estimation for Multi-Domain Task-Oriented Dialog (D19-1)
Copied to clipboard
| Challenge: | Existing methods to learn dialog policy require elaborate design and user goals. |
| Approach: | They propose an algorithm that estimates the reward signal and infers the user goal in dialog sessions. |
| Outcome: | The proposed algorithm achieves higher task success than state-of-the-art models on a multi-domain task-oriented dialog dataset. |
DyBBT: Dynamic Balance via Bandit-inspired Targeting for Dialog Policy with Cognitive Dual Systems (2026.acl-long)
Copied to clipboard
| Challenge: | Task oriented dialog systems often rely on static exploration strategies that do not adapt to dynamic dialog contexts. |
| Approach: | They propose a dialog policy learning framework that formalizes the exploration challenge through a structured cognitive state space C. |
| Outcome: | The proposed framework achieves SOTA performance in success rate, efficiency, and generalization. |
Conversational Graph Grounded Policy Learning for Open-Domain Conversation Generation (2020.acl-main)
Copied to clipboard
| Challenge: | Existing word-level policy models that learn dialog policy and language generation from dialog corpora often lead to degeneration issues where the utterances become ungrammatical or repetitive. |
| Approach: | They propose to represent prior dialog transitions as a graph and learn a CG grounded dialog policy that can foster a more coherent and controllable dialog. |
| Outcome: | The proposed framework is able to learn dialog policy in open-domain multi-turn conversation. |
Target-Guided Open-Domain Conversation (P19-1)
Copied to clipboard
| Challenge: | a new study aims to improve opendomain chat systems by integrating goals and strategy into the system. |
| Approach: | They propose a structured approach that introduces coarse-grained keywords to control intended content of system responses and attains smooth conversation transition through turn-level supervised learning. |
| Outcome: | The proposed system produces meaningful and effective conversations significantly better than other approaches. |
Gradient-Adaptive Policy Optimization: Towards Multi-Objective Alignment of Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Reinforcement learning from human feedback (RLHF) has emerged as a powerful technique for aligning large language models (LLMs) with human preferences. |
| Approach: | They propose a novel algorithm that uses multiple-gradient descent to optimize LLMs with diverse preferences to maximize trade-offs between objectives. |
| Outcome: | The proposed approach incorporates user preferences across different objectives and achieves Pareto solutions that better align with the user’s specific needs. |
Rethinking Supervised Learning and Reinforcement Learning in Task-Oriented Dialogue Systems (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Dialogue policy learning for task-oriented dialogue systems has enjoyed great progress through using reinforcement learning methods. |
| Approach: | They propose a dialogue action decoder and a simulator-free adversarial learning method to improve dialogue agent performance without using reinforcement learning. |
| Outcome: | The proposed methods achieve more stable and higher performance with fewer efforts, such as the domain knowledge required to design a user simulator and the intractable parameter tuning in reinforcement learning. |