Papers by Helen Hastie
CRWIZ: A Framework for Crowdsourcing Real-Time Wizard-of-Oz Dialogues (2020.lrec-1)
Copied to clipboard
| Challenge: | Crowdsourcing platforms such as Amazon Mechanical Turk have been effective for collecting large corpora of task-based and open-domain conversational dialogues, but difficulties arise when task- based dialogues require expert domain knowledge or rapid access to domain-relevant information. |
| Approach: | They propose a framework for collecting real-time Wizard of Oz dialogues through crowdsourcing for collaborative, complex tasks. |
| Outcome: | The proposed framework avoids interactions that breach procedures only known to experts while enabling the capture of a wide variety of interactions. |
A Study of Automatic Metrics for the Evaluation of Natural Language Explanations (2021.eacl-main)
Copied to clipboard
| Challenge: | a lack of transparency is a key issue for robotics and AI. |
| Approach: | They propose to map existing automatic evaluation methods for natural language generation onto explanations. |
| Outcome: | The proposed model shows that embedding-based evaluation methods have higher correlations with human ratings than word-overlap metrics. |