| Challenge: | Using a task-oriented visual dialogue model, we add a decision-making component that decides whether to ask a follow-up question to identify a target referent in an image, or to stop the conversation to make a guess. |
| Approach: | They augment a task-oriented visual dialogue model with a decision-making component that decides whether to ask a follow-up question to identify a target referent in an image, or to stop the conversation to make a guess. |
| Outcome: | The proposed model can be enhanced with a decision-making component that decides whether to ask a follow-up question to identify a target referent in an image, or to stop the conversation to make a guess. |
Similar Papers
Looking for Confirmations: An Effective and Human-Like Visual Dialogue Strategy (2021.emnlp-main)
Copied to clipboard
| Challenge: | State-of-the-art systems generate questions that sound unnatural to humans and are grammatically correct. |
| Approach: | They propose to use beam search re-ranking to generate a model that guides an effective goal-oriented strategy by asking questions that confirm the model’s conjecture about the referent. |
| Outcome: | The proposed model is more natural and effective than beam search decoding without re-ranking on the GuessWhat?! game. |
The Interplay of Task Success and Dialogue Quality: An in-depth Evaluation in Task-Oriented Visual Dialogues (2021.eacl-main)
Copied to clipboard
| Challenge: | chit-chat and task-oriented dialogue models are evaluated on their task success metric, but the best model is usually chosen based on task success. |
| Approach: | They compare models playing different games to find out which one is best . they find that this discrepancy is model- and task-agnostic . |
| Outcome: | The proposed model can generate utterances that are indistinguishable from human dialogues by learning to ground, encode, and decode words that do not occur frequently in the training set. |
What Should I Ask? Using Conversationally Informative Rewards for Goal-oriented Visual Dialog. (P19-1)
Copied to clipboard
| Challenge: | a new study challenges the ability of artificial agents to engage in goal-oriented conversations . goal-orientated visual dialogue is a challenging task since it requires a strategy and contextual information to achieve a goal. |
| Approach: | They propose a goal-oriented visual dialogue system that combines reinforcement learning with regularized information gain. |
| Outcome: | The proposed system outperforms current state-of-the-art models on the GuessWhat?! dataset. |
Saying No is An Art: Contextualized Fallback Responses for Unanswerable Dialogue Queries (2021.acl-short)
Copied to clipboard
| Challenge: | despite advances in task-oriented and chit-chat based dialogue systems, many systems rely on static and unnatural responses. |
| Approach: | They propose a neural approach which generates contextually aware responses to user queries . they perform automatic and manual evaluations to demonstrate the efficacy of the system . |
| Outcome: | The proposed approach generates responses which are contextually aware with the user query and say no to the user. |
Enhancing Visual Dialog Questioner with Entity-based Strategy Learning and Augmented Guesser (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to build a visual dialog (VD) Questioner do not provide explicit guidance for questioner to generate visually related and informative questions. |
| Approach: | They propose a Related entity enhanced Questioner that learns entity-based questioning strategy from human dialogs. |
| Outcome: | The proposed approach achieves state-of-the-art performance on image-guessing task and question diversity. |
Refer, Reuse, Reduce: Generating Subsequent References in Visual and Conversational Contexts (2020.emnlp-main)
Copied to clipboard
| Challenge: | Subsequent references exploit the common ground accumulated by the interlocutors and tend to be shorter and reuse expressions that were effective in previous mentions. |
| Approach: | They propose a model that generates first and subsequent references in visually grounded dialogue . they also implement a reference resolution system to assess the referring effectiveness . |
| Outcome: | The proposed model produces better, more effective referring utterances than one not grounded in the dialogue context. |
Improving Generative Visual Dialog by Answering Diverse Questions (D19-1)
Copied to clipboard
| Challenge: | Prior work on training generative Visual Dialog models with reinforcement learning has shown that this improvement saturates and starts degrading after a few rounds of interaction, and does not lead to a better Visual Dialog model. |
| Approach: | They propose a Q-Bot-A-Bot image-guessing game that allows Q-BOT to ask diverse questions, thus reducing repetitions and enabling A-BOTT to explore a larger state space during RL. |
| Outcome: | The proposed approach improves Q-Bot-A-Bot image-guessing performance but degrades after a few rounds of interaction and does not lead to a better Visual Dialog model. |
The Dialogue Dodecathlon: Open-Domain Knowledge and Image Grounded Conversational Agents (2020.acl-main)
Copied to clipboard
| Challenge: | a set of 12 tasks that measure if a conversational agent can communicate engagingly with personality and empathy, ask questions, answer questions by utilizing knowledge resources, and perceive and converse about images. |
| Approach: | They propose a set of 12 tasks that measure if a conversational agent can communicate engagingly with personality and empathy . they use large dialogue datasets to multi-task and obtain state-of-the-art results . |
| Outcome: | The proposed model improves over a BERT pre-trained model on large dialogue datasets and provides state-of-the-art results on many of the tasks. |
Beyond task success: A closer look at jointly learning to see, ask, and GuessWhat (N19-1)
Copied to clipboard
Ravi Shekhar, Aashish Venkatesh, Tim Baumgärtner, Elia Bruni, Barbara Plank, Raffaella Bernardi, Raquel Fernández
| Challenge: | Existing systems that address the abilities that need to be put to work during conversations are lacking in terms of visual grounding. |
| Approach: | They propose a visually-grounded dialogue state encoder which integrates visual grounding with dialogue system components. |
| Outcome: | The proposed system improves the GuessWhat?! game by combining guessing and asking questions with multi-task learning. |
Asking More Informative Questions for Grounded Retrieval (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing approaches to question generation for interactive retrieval have constrained answer spaces, limiting the amount of information a model can gain in a single turn. |
| Approach: | They propose a method that incorporates presupposition handling into question selection and belief updates. |
| Outcome: | The proposed method increases accuracy over the past state-of-the-art by 14% while resulting in 48% more efficient games in human evaluations. |