Papers with GuessWhat
LEATHER: A Framework for Learning to Generate Human-like Text in Dialogue (2022.findings-aacl)
Copied to clipboard
| Challenge: | Generating coherent, human-like text for dialogue remains a challenge . lack of careful design of rewards can lead to mode-collapse in dialogue . |
| Approach: | They propose a theoretical framework for learning to generate text in dialogue . they propose to use data-shift to develop theoretical guarantees for learners . |
| Outcome: | The proposed framework improves both task-success and human-likeness of the generated text. |
“I’ve Seen Things You People Wouldn’t Believe”: Hallucinating Entities in GuessWhat?! (2021.acl-srw)
Copied to clipboard
| Challenge: | a problem with natural language generation systems is the generation of tokens that are unrelated to the source input. |
| Approach: | They propose two new models to play the GuessWhat?! referential game . they propose to adapt the best visual processing models available to mitigate this issue . |
| Outcome: | The proposed models generate few hallucinations compared to other models available in the literature. |
Learning to Generate Equitable Text in Dialogue from Biased Training Data (2023.acl-long)
Copied to clipboard
| Challenge: | Absence of equitable and inclusive principles can hinder the formation of common ground, which in turn negatively impacts the overall performance of the system. |
| Approach: | They propose to use theories of computational learning to study equitable text generation in dialogues using augmented data to prove formal definitions of equity in text generation and formal connections between human-likeness and learning equity. |
| Outcome: | The proposed model predicts relative-performance of multiple algorithms in generating equitable text as measured by human and automated evaluation. |
The Interplay of Task Success and Dialogue Quality: An in-depth Evaluation in Task-Oriented Visual Dialogues (2021.eacl-main)
Copied to clipboard
| Challenge: | chit-chat and task-oriented dialogue models are evaluated on their task success metric, but the best model is usually chosen based on task success. |
| Approach: | They compare models playing different games to find out which one is best . they find that this discrepancy is model- and task-agnostic . |
| Outcome: | The proposed model can generate utterances that are indistinguishable from human dialogues by learning to ground, encode, and decode words that do not occur frequently in the training set. |
CompGuessWhat?!: A Multi-task Evaluation Framework for Grounded Language Learning (2020.acl-main)
Copied to clipboard
Alessandro Suglia, Ioannis Konstas, Andrea Vanzo, Emanuele Bastianelli, Desmond Elliott, Stella Frank, Oliver Lemon
| Challenge: | Approaches to Grounded Language Learning focus on a single task-based final performance measure which may not depend on desirable properties of the learned hidden representations. |
| Approach: | They propose an evaluation framework for Grounded Language Learning with Attributes based on three sub-tasks: 1) Goal-oriented evaluation; 2) Object attribute prediction evaluation; and 3) Zero-shot evaluation. |
| Outcome: | The proposed framework evaluates the quality of learned representations with respect to attribute grounding. |