Beyond task success: A closer look at jointly learning to see, ask, and GuessWhat (N19-1)
Copied to clipboard
Ravi Shekhar, Aashish Venkatesh, Tim Baumgärtner, Elia Bruni, Barbara Plank, Raffaella Bernardi, Raquel Fernández
| Challenge: | Existing systems that address the abilities that need to be put to work during conversations are lacking in terms of visual grounding. |
| Approach: | They propose a visually-grounded dialogue state encoder which integrates visual grounding with dialogue system components. |
| Outcome: | The proposed system improves the GuessWhat?! game by combining guessing and asking questions with multi-task learning. |
Similar Papers
The Interplay of Task Success and Dialogue Quality: An in-depth Evaluation in Task-Oriented Visual Dialogues (2021.eacl-main)
Copied to clipboard
| Challenge: | chit-chat and task-oriented dialogue models are evaluated on their task success metric, but the best model is usually chosen based on task success. |
| Approach: | They compare models playing different games to find out which one is best . they find that this discrepancy is model- and task-agnostic . |
| Outcome: | The proposed model can generate utterances that are indistinguishable from human dialogues by learning to ground, encode, and decode words that do not occur frequently in the training set. |
Ask No More: Deciding when to guess in referential visual dialogue (C18-1)
Copied to clipboard
| Challenge: | Using a task-oriented visual dialogue model, we add a decision-making component that decides whether to ask a follow-up question to identify a target referent in an image, or to stop the conversation to make a guess. |
| Approach: | They augment a task-oriented visual dialogue model with a decision-making component that decides whether to ask a follow-up question to identify a target referent in an image, or to stop the conversation to make a guess. |
| Outcome: | The proposed model can be enhanced with a decision-making component that decides whether to ask a follow-up question to identify a target referent in an image, or to stop the conversation to make a guess. |
CompGuessWhat?!: A Multi-task Evaluation Framework for Grounded Language Learning (2020.acl-main)
Copied to clipboard
Alessandro Suglia, Ioannis Konstas, Andrea Vanzo, Emanuele Bastianelli, Desmond Elliott, Stella Frank, Oliver Lemon
| Challenge: | Approaches to Grounded Language Learning focus on a single task-based final performance measure which may not depend on desirable properties of the learned hidden representations. |
| Approach: | They propose an evaluation framework for Grounded Language Learning with Attributes based on three sub-tasks: 1) Goal-oriented evaluation; 2) Object attribute prediction evaluation; and 3) Zero-shot evaluation. |
| Outcome: | The proposed framework evaluates the quality of learned representations with respect to attribute grounding. |
Reference-Centric Models for Grounded Collaborative Dialogue (2021.emnlp-main)
Copied to clipboard
| Challenge: | Using a structured referent grounding module, we can effectively ground and inform a partner's utterances to their own context. |
| Approach: | They propose a grounded neural dialogue model that works with people in a partially-observable reference game. |
| Outcome: | The proposed model outperforms state-of-the-art models on a spatial grounding dialogue task and achieves a 20% relative improvement in human evaluations. |
The Dialogue Dodecathlon: Open-Domain Knowledge and Image Grounded Conversational Agents (2020.acl-main)
Copied to clipboard
| Challenge: | a set of 12 tasks that measure if a conversational agent can communicate engagingly with personality and empathy, ask questions, answer questions by utilizing knowledge resources, and perceive and converse about images. |
| Approach: | They propose a set of 12 tasks that measure if a conversational agent can communicate engagingly with personality and empathy . they use large dialogue datasets to multi-task and obtain state-of-the-art results . |
| Outcome: | The proposed model improves over a BERT pre-trained model on large dialogue datasets and provides state-of-the-art results on many of the tasks. |
Achieving Common Ground in Multi-modal Dialogue (2020.acl-tutorials)
Copied to clipboard
| Challenge: | tutorial focuses on three main topic areas: grounding in human-human communication, dialogue systems and multi-modal interactive systems. |
| Approach: | This tutorial examines the use of computational dialogue research to design grounding modules and behaviors in cutting-edge systems. |
| Outcome: | This tutorial examines the results of recent research on grounding in human-human communication . it shows how these results lead to rich and challenging opportunities for doing grounding more flexible and powerful ways . |
Looking for Confirmations: An Effective and Human-Like Visual Dialogue Strategy (2021.emnlp-main)
Copied to clipboard
| Challenge: | State-of-the-art systems generate questions that sound unnatural to humans and are grammatically correct. |
| Approach: | They propose to use beam search re-ranking to generate a model that guides an effective goal-oriented strategy by asking questions that confirm the model’s conjecture about the referent. |
| Outcome: | The proposed model is more natural and effective than beam search decoding without re-ranking on the GuessWhat?! game. |
Imagining Grounded Conceptual Representations from Perceptual Information in Situated Guessing Games (2020.coling-main)
Copied to clipboard
Alessandro Suglia, Antonio Vergari, Ioannis Konstas, Yonatan Bisk, Emanuele Bastianelli, Andrea Vanzo, Oliver Lemon
| Challenge: | Existing models fail to learn multi-modal representations, relying on category labels at inference time. |
| Approach: | They propose a "imagination" module that learns context-aware and category-awful latent embeddings without relying on category labels at inference time. |
| Outcome: | The imagination module outperforms state-of-the-art competitors by 8.26% gameplay accuracy in the CompGuessWhat?! benchmark. |
The PhotoBook Dataset: Building Common Ground through Visually-Grounded Dialogue (P19-1)
Copied to clipboard
| Challenge: | Using the PhotoBook dataset, we investigate shared dialogue history accumulating during conversation . human interlocutors are known to collaboratively establish a shared repository of mutual information during a conversation - this common ground is then used to optimise understanding and communication efficiency. |
| Approach: | They propose a data-collection task formulated as a collaborative game prompting two online participants to refer to images utilising both their visual context and previously established referring expressions. |
| Outcome: | The proposed model takes into account shared information accumulated in a reference chain and is important to resolve later descriptions. |
Sequence-to-Sequence Learning for Task-oriented Dialogue with Dialogue State Representation (C18-1)
Copied to clipboard
| Challenge: | Existing pipeline models for task-oriented dialogue system require explicit modeling of dialogue states and hand-crafted action spaces to query domain-specific knowledge base. |
| Approach: | They propose a framework that leverages the advantages of classic pipeline and sequence-to-sequence models. |
| Outcome: | The proposed framework outperforms baseline models on automatic and human evaluation on a Stanford Multi-turn Multi-domain task-oriented dialogue dataset. |