PLATO-2: Towards Building an Open-Domain Chatbot via Curriculum Learning (2021.findings-acl)
Copied to clipboard
| Challenge: | PLATO-2 is a high-quality open-domain chatbot that can generate one-to-many mappings and improve response quality. |
| Approach: | They propose a curriculum learning process to build a high-quality open-domain chatbot . they use a coarse-grained generation model and latent variables to train a generative model . |
| Outcome: | The proposed model improves on Chinese and English data and can generate diverse responses and select the best response. |
Similar Papers
Recipes for Building an Open-Domain Chatbot (2021.eacl-main)
Copied to clipboard
Stephen Roller, Emily Dinan, Naman Goyal, Da Ju, Mary Williamson, Yinhan Liu, Jing Xu, Myle Ott, Eric Michael Smith, Y-Lan Boureau, Jason Weston
| Challenge: | Existing work shows that scaling models in the number of parameters and the size of the data they are trained on gives improved results, but other factors are important. |
| Approach: | They propose to build open-domain chatbots that can be scaled to improve their performance . they use a blend of cognitive and cognitive skills to build a model that combines these skills . |
| Outcome: | The proposed models outperform existing approaches in multi-turn dialogue on engagingness and humanness measurements. |
Deep Chit-Chat: Deep Learning for ChatBots (D18-3)
Copied to clipboard
| Challenge: | tutorial focuses on building conversational models with deep learning approaches for chatbots. |
| Approach: | This tutorial focuses on building conversational models with deep learning approaches for chatbots. |
| Outcome: | The tutorial summarizes the fundamental challenges in modeling open domain dialogues . it also covers some new trends of research of chatbots - such as how to "control" conversations with specific information . |
PLATO: Pre-trained Dialogue Generation Model with Discrete Latent Variable (2020.acl-main)
Copied to clipboard
| Challenge: | Existing pre-training models for dialogue generation have been proven effective for a wide range of tasks. |
| Approach: | They propose a dialogue generation pre-training framework that leverages bi-directional context and uni-directional characteristic of language generation. |
| Outcome: | The proposed framework is superior to existing models on three publicly available datasets. |
PLATO-XL: Exploring the Large-scale Pre-training of Dialogue Generation (2022.findings-aacl)
Copied to clipboard
Siqi Bao, Huang He, Fan Wang, Hua Wu, Haifeng Wang, Wenquan Wu, Zhihua Wu, Zhen Guo, Hua Lu, Xinxian Huang, Xin Tian, Xinchao Xu, Yingzhan Lin, Zheng-Yu Niu
| Challenge: | Experimental results show PLATO-XL achieves state-of-the-art results across multiple conversational tasks. |
| Approach: | They propose to train PLATO-XL models with up to 11 billion parameters, trained on Chinese and English social media conversations. |
| Outcome: | The proposed model achieves state-of-the-art on multiple conversational tasks, verifying its potential as a foundation model of conversational AI. |
Book2Dial: Generating Teacher Student Interactions from Textbooks for Cost-Effective Development of Educational Chatbots (2024.findings-acl)
Copied to clipboard
| Challenge: | Educational chatbots are a promising tool for assisting student learning, but high-quality data is difficult to obtain due to privacy concerns. |
| Approach: | They propose a framework for generating synthetic teacher-student interactions grounded in a set of textbooks and propose to open-source their results. |
| Outcome: | The proposed framework captures a key aspect of learning interactions where curious students with partial knowledge ask teachers questions about the material in the textbook. |
Grammar Control in Dialogue Response Generation for Language Learning Chatbots (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing language learning chatbots and research on second language acquisition benefit from these affordances. |
| Approach: | They ground a dialogue response generation model in a pedagogical repository of grammar skills and evaluate prompting, fine-tuning, and decoding strategies for grammar-controlled dialogue response generators. |
| Outcome: | The proposed model outperforms GPT-3.5 when tolerating minor response quality losses and predicts grammar-controlled responses to support grammar acquisition adapted to learner proficiency. |
In-sample Curriculum Learning by Sequence Completion for Natural Language Generation (2023.acl-long)
Copied to clipboard
| Challenge: | Existing work on curriculum learning rely on task-specific expertise and cannot generalize to different tasks. |
| Approach: | They propose to do in-sample curriculum learning for natural language generation tasks using human-crafted rules and a numeric score for each sample based on domain expertise to rank the model. |
| Outcome: | The proposed learning strategy generalizes well to different tasks and achieves significant improvements over baselines. |
Data Collection and End-to-End Learning for Conversational AI (D19-2)
Copied to clipboard
| Challenge: | tutorial aims to familiarise research community with recent advances in statistical dialogue systems . focus of tutorial is on learning end-to-end from data and their relation to more common modular systems. |
| Approach: | This tutorial aims to familiarise the research community with the latest advances in statistical dialogue systems . the focus of the tutorial is on recently introduced end-to-end learning for dialogue systems and their relation to more common modular systems. |
| Outcome: | This tutorial aims to familiarise the research community with the recent advances in statistical dialogue systems for open-domain and task-based dialogue paradigms. |
Polite Chatbot: A Text Style Transfer Application (2023.eacl-srw)
Copied to clipboard
| Challenge: | Creating polite chatbots requires complex setups that require reinforcement learning to produce coherent responses. |
| Approach: | They propose a polite chatbot that can generate coherent responses to given contexts by using a model that transfers neutral sentences into polite ones and trains a dialogue model. |
| Outcome: | The proposed method outperforms baselines in producing polite dialogue responses while staying competitive in terms of coherent to the given context. |
Towards Boosting the Open-Domain Chatbot with Human Feedback (2023.acl-long)
Copied to clipboard
| Challenge: | Existing frameworks for pre-training open-domain dialogue models with social media comments generate coherent replies but have difficulties producing engaging responses. |
| Approach: | They propose a framework to boost the open-domain chatbot by leveraging human feedback and annotating the model's candidate responses. |
| Outcome: | The proposed framework boosts the open-domain chatbot by leveraging human demonstrated responses and leveraging the implicit preference in the data collection process. |