| Challenge: | Existing studies focus on grounding conversational agents on text-only corpora, but they lack the perception ability to our physical world. |
| Approach: | They propose to ground conversational agents on images retrieved from large-scale image indexes . they propose to use visual knowledge to generate informative responses based on the extracted knowledge . |
| Outcome: | The proposed agent outperforms state-of-the-art methods on automatic metrics and human evaluation. |
Similar Papers
Image-Chat: Engaging Grounded Conversations (2020.acl-main)
Copied to clipboard
| Challenge: | In order for machines to communicate with humans, they must understand the natural things that humans say about the world they live in and respond in kind. |
| Approach: | They propose to fuse a set of neural architectures using image and text representations to achieve this goal. |
| Outcome: | The proposed model performs well on the Image-Chat task and humans prefer it 47.7% of the time. |
The Dialogue Dodecathlon: Open-Domain Knowledge and Image Grounded Conversational Agents (2020.acl-main)
Copied to clipboard
| Challenge: | a set of 12 tasks that measure if a conversational agent can communicate engagingly with personality and empathy, ask questions, answer questions by utilizing knowledge resources, and perceive and converse about images. |
| Approach: | They propose a set of 12 tasks that measure if a conversational agent can communicate engagingly with personality and empathy . they use large dialogue datasets to multi-task and obtain state-of-the-art results . |
| Outcome: | The proposed model improves over a BERT pre-trained model on large dialogue datasets and provides state-of-the-art results on many of the tasks. |
Multi-Modal Open-Domain Dialogue (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent work in open-domain conversational agents has demonstrated that significant improvements in humanness and user preference can be achieved via massive scaling in both pre-training data and model size. |
| Approach: | They combine open-domain dialogue agents with vision models to investigate human preferences and humanness. |
| Outcome: | The proposed model outperforms existing models in multi-modal dialogue while performing as well as its predecessor (text-only) BlenderBot. |
Ask No More: Deciding when to guess in referential visual dialogue (C18-1)
Copied to clipboard
| Challenge: | Using a task-oriented visual dialogue model, we add a decision-making component that decides whether to ask a follow-up question to identify a target referent in an image, or to stop the conversation to make a guess. |
| Approach: | They augment a task-oriented visual dialogue model with a decision-making component that decides whether to ask a follow-up question to identify a target referent in an image, or to stop the conversation to make a guess. |
| Outcome: | The proposed model can be enhanced with a decision-making component that decides whether to ask a follow-up question to identify a target referent in an image, or to stop the conversation to make a guess. |
Bootstrapping a Neural Conversational Agent with Dialogue Self-Play, Crowdsourcing and On-Line Reinforcement Learning (N18-3)
Copied to clipboard
| Challenge: | End-to-end neural models for conversational agents require large corpus of dialogues to learn effectively. |
| Approach: | They propose a method for building an agent for arbitrary tasks by combining dialogue self-play and crowd-sourcing. |
| Outcome: | The proposed approach can be quickly bootstrapped to deploy in front of users and further optimized via interactive learning from actual users. |
Reference-Centric Models for Grounded Collaborative Dialogue (2021.emnlp-main)
Copied to clipboard
| Challenge: | Using a structured referent grounding module, we can effectively ground and inform a partner's utterances to their own context. |
| Approach: | They propose a grounded neural dialogue model that works with people in a partially-observable reference game. |
| Outcome: | The proposed model outperforms state-of-the-art models on a spatial grounding dialogue task and achieves a 20% relative improvement in human evaluations. |
DIALOGPT : Large-Scale Generative Pre-training for Conversational Response Generation (2020.acl-demos)
Copied to clipboard
Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, Bill Dolan
| Challenge: | DIALOGPT is a large, tunable neural conversational response generation model . trained on 147M conversation-like exchanges extracted from Reddit comment chains . |
| Approach: | They present a large, tunable neural conversational response generation model, DIALOGPT . the model is trained on 147M conversation-like exchanges extracted from Reddit comment chains . |
| Outcome: | The proposed model can generate more relevant, contentful and context-consistent responses than baseline systems. |
You Truly Understand What I Need : Intellectual and Friendly Dialog Agents grounding Persona and Knowledge (2022.findings-emnlp)
Copied to clipboard
Jungwoo Lim, Myunghoon Kang, Yuna Hur, Seung Won Jeong, Jinsung Kim, Yoonna Jang, Dongyub Lee, Hyesung Ji, DongHoon Shin, Seungryong Kim, Heuiseok Lim
| Challenge: | Existing models that ground knowledge and persona at the same time are limited, leading to hallucination and a passive way of using personas. |
| Approach: | They propose a conversational agent that grounds external knowledge and persona simultaneously and a retrieval augmented generation model that generates utterances with lesser hallucination and more engagingness. |
| Outcome: | The proposed agent generates the utterance with lesser hallucination and more engagingness utilizing retrieval augmented generation with knowledge-persona enhanced query. |
Grounding in social media: An approach to building a chit-chat dialogue model (2022.naacl-srw)
Copied to clipboard
| Challenge: | Existing open-domain dialogue models fail to capture and utilize external knowledge, leading to repetitive or generic responses to unseen utterances. |
| Approach: | They propose to use social media comments to improve the raw conversation ability of open-domain dialogue systems. |
| Outcome: | The proposed model improves the raw conversation ability of open-domain dialogue systems by mimicking human responses through casual interactions found on social media. |
Interactive Language Acquisition with One-shot Visual Concept Learning through a Conversational Game (P18-1)
Copied to clipboard
| Challenge: | supervised language learning is limited by the ability of capturing mainly the statistics of training data. |
| Approach: | They propose to use conversational games to train agents to use new knowledge . they propose to mimic and reinforce conversational game and use it in one-shot fashion . |
| Outcome: | The proposed approach is able to acquire information by asking questions about novel objects and use the just-learned knowledge in subsequent conversations in a one-shot fashion. |