| Challenge: | Reinforcement learning (RL) is an attractive solution for task-oriented dialog systems . but extending RL-based systems to handle new intents and slots requires a system redesign . |
| Approach: | They propose a teacher-student framework to extend RL-based dialog systems . they propose to specify constraints held in the new dialog manager . |
| Outcome: | The proposed framework makes no assumption about unsupported intents and slots, making it possible to improve RL-based systems incrementally. |
Similar Papers
Building Task-Oriented Visual Dialog Systems Through Alternative Optimization Between Dialog Policy and Language Generation (D19-1)
Copied to clipboard
| Challenge: | Current approaches to visual dialog learning involve an end-to-end framework that maps the multi-modal context to a deep vector and in order to decode a natural dialog response. |
| Approach: | They propose a framework that trains a RL policy for image guessing and a seq2seq model to improve dialog quality. |
| Outcome: | The proposed framework achieves state-of-the-art performance on a guessWhich task . it can be applied to a wide range of tasks including assisting blind people . |
Human-centric dialog training via offline reinforcement learning (2020.emnlp-main)
Copied to clipboard
Natasha Jaques, Judy Hanwen Shen, Asma Ghandeharioun, Craig Ferguson, Agata Lapedriza, Noah Jones, Shixiang Gu, Rosalind Picard
| Challenge: | a novel offline RL method can train dialog models to produce better conversations without the risk of humans teaching it harmful chat behaviors. |
| Approach: | They develop offline reinforcement learning algorithms that use human feedback to train dialog models . they use language similarity, laughter, sentiment, and more to identify positive feedback . |
| Outcome: | The proposed method improves on existing methods with 80 users in an open-domain setting. |
Scheduled Dialog Policy Learning: An Automatic Curriculum Learning Framework for Task-oriented Dialog System (2021.findings-acl)
Copied to clipboard
| Challenge: | et al., 2013) show that dialog policy learning is an important component of the task-oriented dialogue system. |
| Approach: | They propose a framework that integrates curriculum learning and policy optimization . they propose to train dialog agents from easy dialogues to complex ones . |
| Outcome: | The proposed framework outperforms the state-of-the-art model on multi-task dialogues. |
From Problem-Solving to Teaching Problem-Solving: Aligning LLMs with Pedagogy using Reinforcement Learning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) are often optimized for direct question-answering, but their effectiveness is often undermined by strategically withholding answers. |
| Approach: | They propose an online reinforcement learning-based alignment framework that can quickly adapt LLMs into effective tutors using simulated student-tutor interactions. |
| Outcome: | The proposed model outperforms proprietary models like LearnLM and can be used to enhance interpretability and pedagogical quality. |
How to Build User Simulators to Train RL-based Dialog Systems (D19-1)
Copied to clipboard
| Challenge: | RL-based dialog systems require interaction with the environment and obtaining real human users to interact with the system is time-consuming and labor-intensive. |
| Approach: | They propose a method to standardize user simulator building to compare dialog system quality using the same set of user simulators. |
| Outcome: | The proposed method can be used by the community to compare dialog system quality using the same set of user simulators fairly. |
Feudal Reinforcement Learning for Dialogue Management in Large Domains (N18-2)
Copied to clipboard
Iñigo Casanueva, Paweł Budzianowski, Pei-Hao Su, Stefan Ultes, Lina M. Rojas-Barahona, Bo-Hsiang Tseng, Milica Gašić
| Challenge: | Reinforcement learning (RL) is a promising approach to model dialogue policy optimisation but fails to scale to large domains due to the curse of dimensionality. |
| Approach: | They propose a novel approach to dialogue policy optimisation using reinforcement learning . they propose to decompose the decision into two steps using a domain ontology . |
| Outcome: | The proposed architecture outperforms state-of-the-art in several dialogue domains without any additional reward signal. |
Imperfect also Deserves Reward: Multi-Level and Sequential Reward Modeling for Better Dialog Management (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing research on taskoriented dialog systems mainly includes pipeline and end-to-end methods due to its non-differentiable nature. |
| Approach: | They propose a multi-level reward modeling approach that factorizes a reward into a three-level hierarchy: domain, act, and slot. |
| Outcome: | The proposed approach significantly improves performance and speed of training in a wide range of dialog systems. |
MinTL: Minimalist Transfer Learning for Task-Oriented Dialogue Systems (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to learn dialogue state tracking and response generation are time-intensive and not transferable between domains. |
| Approach: | They propose a transfer learning framework that allows efficient dialogue state tracking with a minimal generation length. |
| Outcome: | The proposed framework improves the inference efficiency and improves state-of-the-art results on multi-domain multi-tasking systems. |
Rewarding What Matters: Step-by-Step Reinforcement Learning for Task-Oriented Dialogue (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing RL methods focus on generation tasks while neglecting dialogue state tracking (DST) for understanding. |
| Approach: | They propose a method that integrates RL into both understanding and generation tasks by introducing step-by-step rewards throughout the token generation. |
| Outcome: | The proposed approach achieves state-of-the-art results on three widely used datasets. |
Phrase-Level Action Reinforcement Learning for Neural Dialog Response Generation (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for dialog agent training lack a robust action space for entangled information, which can cause bias and deviate from natural human language. |
| Approach: | They propose phrase-level action reinforcement learning which allows the model to alter the sentence structure and content with the sequential action selection. |
| Outcome: | The proposed model achieves competitive results with state-of-the-art models on the MultiWOZ dataset, indicating that it is effective for solving task-oriented dialogs. |