Papers by Blair Johnston
Meta-Learning Adaptive Knowledge Distillation for Efficient Biomedical Natural Language Processing (2022.findings-aacl)
Copied to clipboard
| Challenge: | Existing knowledge distillation methods have been proposed to reduce the size of large models for biomedical natural language processing tasks. |
| Approach: | They propose a meta-learning approach which adaptively learns parameters that enable optimal rate of knowledge exchange between teacher and student models from the distillation data during knowledge distillation. |
| Outcome: | The proposed method improves the performance of knowledge distillation methods on two biomedical natural language processing tasks. |