Stephen Roller, Emily Dinan, Naman Goyal, Da Ju, Mary Williamson, Yinhan Liu, Jing Xu, Myle Ott, Eric Michael Smith, Y-Lan Boureau, Jason Weston
| Challenge: | Existing work shows that scaling models in the number of parameters and the size of the data they are trained on gives improved results, but other factors are important. |
| Approach: | They propose to build open-domain chatbots that can be scaled to improve their performance . they use a blend of cognitive and cognitive skills to build a model that combines these skills . |
| Outcome: | The proposed models outperform existing approaches in multi-turn dialogue on engagingness and humanness measurements. |
Similar Papers
Deep Chit-Chat: Deep Learning for ChatBots (D18-3)
Copied to clipboard
| Challenge: | tutorial focuses on building conversational models with deep learning approaches for chatbots. |
| Approach: | This tutorial focuses on building conversational models with deep learning approaches for chatbots. |
| Outcome: | The tutorial summarizes the fundamental challenges in modeling open domain dialogues . it also covers some new trends of research of chatbots - such as how to "control" conversations with specific information . |
PLATO-2: Towards Building an Open-Domain Chatbot via Curriculum Learning (2021.findings-acl)
Copied to clipboard
| Challenge: | PLATO-2 is a high-quality open-domain chatbot that can generate one-to-many mappings and improve response quality. |
| Approach: | They propose a curriculum learning process to build a high-quality open-domain chatbot . they use a coarse-grained generation model and latent variables to train a generative model . |
| Outcome: | The proposed model improves on Chinese and English data and can generate diverse responses and select the best response. |
Building a Role Specified Open-Domain Dialogue System Leveraging Large-Scale Language Models (2022.naacl-main)
Copied to clipboard
| Challenge: | Recent large-scale language models have produced human-like responses in open-domain dialogue systems. |
| Approach: | They propose a framework for imposing roles on open-domain dialogue systems . they use few-shot learning to build a Korean dialogue dataset from scratch . |
| Outcome: | The proposed framework meets role specifications while maintaining conversational abilities. |
MovieChats: Chat like Humans in a Closed Domain (2020.emnlp-main)
Copied to clipboard
| Challenge: | Currently, open-domain chatbots are far from satisfactory. |
| Approach: | They propose a unified, readily scalable neural approach which reconciles all subtasks like intent prediction and knowledge retrieval. |
| Outcome: | The proposed approach outperforms commercial systems replying on complex rules on static and interactive tests and shows that the results are remarkably good. |
LiveChat: A Large-Scale Personalized Dialogue Dataset Automatically Constructed from Live Streaming (2023.acl-long)
Copied to clipboard
| Challenge: | a recent study shows that open-domain dialogue systems are not able to perform well in fast-growing scenarios such as live streaming due to the domain gap between online-post constructed data and those required in downstream conversational tasks. |
| Approach: | They propose to train a conversational agent based on large social media datasets with multiple domains to improve response in live streaming scenarios. |
| Outcome: | The proposed model improves response modeling and addressee recognition in live open-domain scenarios. |
A Recipe For Building a Compliant Real Estate Chatbot (2025.coling-industry)
Copied to clipboard
| Challenge: | In recent years, there has been significant effort to align large language models with human preferences. |
| Approach: | They propose to use a general instruction-following dataset to generate a chatbot that can be used without perpetuating discriminatory practices like steering and redlining. |
| Outcome: | The proposed model can match closed-source models like GPT-4o while being safer and more compliant. |
Multi-Modal Open-Domain Dialogue (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent work in open-domain conversational agents has demonstrated that significant improvements in humanness and user preference can be achieved via massive scaling in both pre-training data and model size. |
| Approach: | They combine open-domain dialogue agents with vision models to investigate human preferences and humanness. |
| Outcome: | The proposed model outperforms existing models in multi-modal dialogue while performing as well as its predecessor (text-only) BlenderBot. |
Deep Learning for Conversational AI (N18-6)
Copied to clipboard
| Challenge: | Spoken Dialogue Systems (SDS) have great commercial potential . the advent of deep learning has led to significant advances in this area of NLP research . |
| Approach: | This tutorial will introduce researchers to the pipeline framework for modelling goal-oriented dialogue systems. |
| Outcome: | This tutorial will familiarise researchers with the latest advances in spoken dialogue systems . the aim of the course is to encourage dialogue research in the NLP community . |
Can You Put it All Together: Evaluating Conversational Agents’ Ability to Blend Skills (2020.acl-main)
Copied to clipboard
| Challenge: | Existing work has focused on learning specific qualities of conversational agents, but it remains unclear how to combine them. |
| Approach: | They propose to combine models trained towards isolated capabilities with multi-task training to improve conversation performance. |
| Outcome: | The proposed dataset compares models trained towards isolated capabilities with models trained on a single skill. |
Enhancing Chat Language Models by Scaling High-quality Instructional Conversations (2023.emnlp-main)
Copied to clipboard
| Challenge: | a recent study validates the effectiveness of chat language models by fine-tuning instruction data. |
| Approach: | They propose to use a large-scale dataset of instructional conversations to fine-tune a conversational model on instruction data. |
| Outcome: | The proposed model outperforms open-source models in key metrics including scale, average length, diversity, coherence, etc. |