Papers by Mark Perera
Consultation Checklists: Standardising the Human Evaluation of Medical Note Generation (2022.emnlp-industry)
Copied to clipboard
Aleksandar Savkov, Francesco Moramarco, Alex Papadopoulos Korfiatis, Mark Perera, Anya Belz, Ehud Reiter
| Challenge: | Existing studies have shown that note generation is difficult due to subjective nature of many aspects of output quality. |
| Approach: | They propose a protocol that aims to increase objectivity by grounding evaluations in Consultation Checklists, which are created in a preliminary step and then used as a common point of reference during quality assessment. |
| Outcome: | The proposed protocol shows that the evaluations produced in the study are more objective than the original human note. |
Human Evaluation and Correlation with Automatic Metrics in Consultation Note Generation (2022.acl-long)
Copied to clipboard
Francesco Moramarco, Alex Papadopoulos Korfiatis, Mark Perera, Damir Juric, Jack Flann, Ehud Reiter, Anya Belz, Aleksandar Savkov
| Challenge: | Recent studies suggest that note generation systems can be used to generate clinical consultation notes from the verbatim transcript of the consultation. |
| Approach: | They propose to use machine learning to generate consultation notes from the verbatim transcript of the consultation to evaluate their effectiveness. |
| Outcome: | The proposed model performs better than common model-based metrics like BertScore and is open-sourced. |
User-Driven Research of Medical Note Generation Software (2022.naacl-main)
Copied to clipboard
Tom Knoll, Francesco Moramarco, Alex Papadopoulos Korfiatis, Rachel Young, Claudia Ruffini, Mark Perera, Christian Perstl, Ehud Reiter, Anya Belz, Aleksandar Savkov
| Challenge: | Existing studies on how NLP systems could be used in clinical practice focus on technical difficulties and usability challenges involved in implementing them. |
| Approach: | They propose to use Speech Recognition to transcribe the audio of a medical consultation and then to train sequence-to-sequence models to summarise the transcript into a consultation note. |
| Outcome: | The proposed system generates notes in real time during a doctor-patient consultation and is able to capture the salient points of a consultation . the proposed system is based on three rounds of user studies in a live telehealth clinic and identifies a number of clinical use cases that could prove challenging for the system. |