Multi-Agent Task-Oriented Dialog Policy Learning with Role-Aware Reward Decomposition (2020.acl-main)
Copied to clipboard
| Challenge: | Many studies have applied reinforcement learning to train a dialog policy . but modeling a real-world user simulator is challenging and requires domain expertise . |
| Approach: | They propose to build dialog policies with two agents as dialog agents to avoid building a user simulator beforehand. |
| Outcome: | The proposed method can build a system policy and a user policy simultaneously . it can achieve high task success rate through conversational interaction . |
Similar Papers
A Collaborative Multi-agent Reinforcement Learning Framework for Dialog Action Decomposition (2021.emnlp-main)
Copied to clipboard
| Challenge: | Currently, most reinforcement learning methods for dialog policy learning train a centralized agent that selects a predefined joint action concatenating domain name, intent type, and slot name. |
| Approach: | They propose a hierarchical multi-agent framework in which each part of the action is led by a different agent and a joint optimization process that makes agents can exchange their policy information. |
| Outcome: | The proposed framework reduces labor costs for action templates and decreases the size of the action space for each agent. |
Guided Dialog Policy Learning: Reward Estimation for Multi-Domain Task-Oriented Dialog (D19-1)
Copied to clipboard
| Challenge: | Existing methods to learn dialog policy require elaborate design and user goals. |
| Approach: | They propose an algorithm that estimates the reward signal and infers the user goal in dialog sessions. |
| Outcome: | The proposed algorithm achieves higher task success than state-of-the-art models on a multi-domain task-oriented dialog dataset. |
Rethinking Supervised Learning and Reinforcement Learning in Task-Oriented Dialogue Systems (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Dialogue policy learning for task-oriented dialogue systems has enjoyed great progress through using reinforcement learning methods. |
| Approach: | They propose a dialogue action decoder and a simulator-free adversarial learning method to improve dialogue agent performance without using reinforcement learning. |
| Outcome: | The proposed methods achieve more stable and higher performance with fewer efforts, such as the domain knowledge required to design a user simulator and the intractable parameter tuning in reinforcement learning. |
Few-Shot Structured Policy Learning for Multi-Domain and Multi-Task Dialogues (2023.findings-eacl)
Copied to clipboard
| Challenge: | Reinforcement learning is widely adopted to model dialogue managers in task-oriented dialogues, but the user simulator provided by state-of-the-art dialogue frameworks are only rough approximations of human behaviour. |
| Approach: | They propose to use structured policies to improve sample efficiency when learning on multi-domain and multi-task environments. |
| Outcome: | The proposed policies improve sample efficiency and performance on multi-domain and multi-task environments. |
Guided Dialogue Policy Learning without Adversarial Learning in the Loop (2020.findings-emnlp)
Copied to clipboard
Ziming Li, Sungjin Lee, Baolin Peng, Jinchao Li, Julia Kiseleva, Maarten de Rijke, Shahin Shayandeh, Jianfeng Gao
| Challenge: | Reinforcement learning methods suffer from sparse and unstable reward signals . alternating training of dialogue agent and reward model can get stuck in local optima . |
| Approach: | They propose to decompose adversarial training into two steps to improve dialogue policy learning. |
| Outcome: | The proposed method achieves remarkable task success rate using both on-policy and off-poly reinforcement learning methods. |
Retrieve & Memorize: Dialog Policy Learning with Multi-Action Memory (2021.findings-acl)
Copied to clipboard
| Challenge: | Recent years have seen a rapid growth of interest in building task-oriented dialogue systems. |
| Approach: | They propose a retrieve-and-memorize framework to deal with unbalanced distribution of system actions in dialogue datasets. |
| Outcome: | The proposed framework achieves competitive performance among state-of-the-art models on a large-scale task-oriented dialogue dataset. |
Learning Goal-oriented Dialogue Policy with opposite Agent Awareness (2020.aacl-main)
Copied to clipboard
| Challenge: | Existing approaches for goal-oriented dialogue policy learning focus on the target agent policy and treat the opposite agent policy as part of the environment. |
| Approach: | They propose a framework for policy learning in goal-oriented dialogues that uses the opposite agent's policy estimation to improve the target agent by regarding it as part of the target policy. |
| Outcome: | The proposed framework shows superior performance over state-of-the-art models on cooperative and competitive dialogue tasks. |
Scheduled Dialog Policy Learning: An Automatic Curriculum Learning Framework for Task-oriented Dialog System (2021.findings-acl)
Copied to clipboard
| Challenge: | et al., 2013) show that dialog policy learning is an important component of the task-oriented dialogue system. |
| Approach: | They propose a framework that integrates curriculum learning and policy optimization . they propose to train dialog agents from easy dialogues to complex ones . |
| Outcome: | The proposed framework outperforms the state-of-the-art model on multi-task dialogues. |
End-to-End Learning of Task-Oriented Dialogs (N18-4)
Copied to clipboard
| Challenge: | Dissertation addresses the limitations of conventional task-oriented dialog systems . conventions of such systems include a complex pipeline and dialog state tracking . |
| Approach: | They propose a neural network based dialog system that can robustly track dialog state . they propose offline training and online interactive learning methods to improve efficiency . |
| Outcome: | The proposed system can track dialog state, interface with knowledge bases, and integrate structured query results into system responses to successfully complete task-oriented dialog. |
Learning Efficient Dialogue Policy from Demonstrations through Shaping (2020.acl-main)
Copied to clipboard
| Challenge: | Using reinforcement learning to learn dialogue policy requires a large volume of interactions with users. |
| Approach: | They propose a task-oriented dialogue agent that efficiently learns dialogue policy from demonstrations . they use an imitation model to distill knowledge from demonstration and reward shaping . |
| Outcome: | The proposed agent efficiently learns dialogue policy from demonstrations through policy shaping and reward shaping. |