Bootstrapped Policy Learning for Task-oriented Dialogue through Goal Shaping (2024.emnlp-main)
Copied to clipboard
| Challenge: | Despite the promise of reinforcement learning, achieving seamless knowledge transitions in complex dialogue environments is difficult. |
| Approach: | They propose a Bootstrapped Policy Learning framework which adaptively tailors progressively challenging subgoal curriculum for each complex goal through goal shaping. |
| Outcome: | The proposed framework has shown to be effective across four publicly available datasets with different difficulty levels. |
Similar Papers
A Versatile Adaptive Curriculum Learning Framework for Task-oriented Dialogue Policy Learning (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing training paradigms for dialogue policy learning with brute-force random sampling are expensive and lack reliable evaluation of difficulty scores. |
| Approach: | They propose a flexible adaptive curriculum learning framework that integrates curriculum learning with a generic global curriculum. |
| Outcome: | The proposed framework improves learning performance and efficiency on three public dialogue datasets. |
Subgoal Discovery for Hierarchical Dialogue Policy Learning (D18-1)
Copied to clipboard
| Challenge: | Existing methods to develop dialogue agents for complex tasks require sparse reward signals. |
| Approach: | They propose a divide-and-conquer approach that exploits the hidden structure of a task . they use subgoals to divide a goal-oriented task into simpler subgoal sets . |
| Outcome: | The proposed approach performs competitively against state-of-the-art methods that require human-defined subgoals. |
One Planner To Guide Them All ! Learning Adaptive Conversational Planners for Goal-oriented Dialogues (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for goal-oriented dialogues involve training separate models for specific combinations of objectives, leading to computational and scalability issues. |
| Approach: | They propose a new dialogue policy method that can adapt to varying objective preferences at inference time without retraining. |
| Outcome: | The proposed method can adapt to varying objective preferences at inference time without retraining. |
Learning Efficient Dialogue Policy from Demonstrations through Shaping (2020.acl-main)
Copied to clipboard
| Challenge: | Using reinforcement learning to learn dialogue policy requires a large volume of interactions with users. |
| Approach: | They propose a task-oriented dialogue agent that efficiently learns dialogue policy from demonstrations . they use an imitation model to distill knowledge from demonstration and reward shaping . |
| Outcome: | The proposed agent efficiently learns dialogue policy from demonstrations through policy shaping and reward shaping. |
Scheduled Dialog Policy Learning: An Automatic Curriculum Learning Framework for Task-oriented Dialog System (2021.findings-acl)
Copied to clipboard
| Challenge: | et al., 2013) show that dialog policy learning is an important component of the task-oriented dialogue system. |
| Approach: | They propose a framework that integrates curriculum learning and policy optimization . they propose to train dialog agents from easy dialogues to complex ones . |
| Outcome: | The proposed framework outperforms the state-of-the-art model on multi-task dialogues. |
JoTR: A Joint Transformer and Reinforcement Learning Framework for Dialogue Policy Learning (2024.lrec-main)
Copied to clipboard
Wai-Chung Kwan, Huimin Wang, Hongru Wang, Zezhong Wang, Bin Liang, Xian Wu, Yefeng Zheng, Kam-Fai Wong
| Challenge: | Dialogue policy learning (DPL) aims to determine an abstract representation (also known as action) to guide what the response should be. |
| Approach: | They propose a joint Transformer-based model that generates a token-grained policy that allows more dynamic dialogue action generation without the need for predefined action candidates. |
| Outcome: | The proposed model outperforms existing models showing improvements of 9% and 13% in success rate and 34% and 37% in diversity of dialogue actions across two benchmark dialogue modeling tasks. |
Experience as Source for Anticipation and Planning: Experiential Policy Learning for Target-driven Recommendation Dialogues (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to enhance dialogues lack the ability to anticipate user interactions . current approaches lack the capability to anticipate past interactions and to neglect past experiences . |
| Approach: | They propose a framework for enhancing dialogue anticipation with an experiential scoring function that estimates dialogue state potential using similar past interactions stored in long-term memory. |
| Outcome: | Experiments on two datasets show the framework is superior and effective . tree-structured EPL assesses past dialogue states with LLMs and MCTS . |
Learning Goal-oriented Dialogue Policy with opposite Agent Awareness (2020.aacl-main)
Copied to clipboard
| Challenge: | Existing approaches for goal-oriented dialogue policy learning focus on the target agent policy and treat the opposite agent policy as part of the environment. |
| Approach: | They propose a framework for policy learning in goal-oriented dialogues that uses the opposite agent's policy estimation to improve the target agent by regarding it as part of the target policy. |
| Outcome: | The proposed framework shows superior performance over state-of-the-art models on cooperative and competitive dialogue tasks. |
Deep Reinforcement Learning with Hierarchical Action Exploration for Dialogue Generation (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing approaches to improve dialogues with random sampling are inefficient due to the large number of eligible responses with high action values. |
| Approach: | They propose a dual-granularity Q-function that extracts actions based on a grained hierarchy . they use offline RL and learn from multiple reward functions designed to capture emotional nuances in human interactions. |
| Outcome: | The proposed approach outperforms baselines across automatic metrics and human evaluations. |
Deep Reinforcement Learning-based Dialogue Policy with Graph Convolutional Q-network (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for deep reinforcement learning lack the ability to learn the relationship between dialogue states and actions. |
| Approach: | They propose a graph-structured dialogue policy framework for task-oriented dialogue systems that uses bipartite graphs to construct two different bipartites and generate user-related and knowledge-related subgraphs. |
| Outcome: | The proposed framework significantly improves the effectiveness and stability of dialogue policies. |