Papers by Chang-Fu Kuo
exBERT: Extending Pre-trained Models with Domain-specific Vocabulary Under Constrained Training Resources (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to train pre-trained models with limited corpus and computational resources are limited by the complexity of the training resources. |
| Approach: | They propose a method to extend BERT pre-trained models from a general domain to a new pre-train model for a specific domain with a different additive vocabulary. |
| Outcome: | The proposed method outperforms existing methods on biomedical benchmark tasks using the MTL-Bioinformatics-2016 dataset. |