Papers by Dayeon Yim
Ignore Me But Don’t Replace Me: Utilizing Non-Linguistic Elements for Pretraining on the Cybersecurity Domain (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods to train domain expertise for cybersecurity text domains are expensive to train and run. |
| Approach: | They propose to use pretraining methods to account for non-linguistic elements in cybersecurity texts and evaluate their effectiveness through downstream tasks and probing tasks. |
| Outcome: | The proposed strategy outperforms the commonly taken approach of replacing NLEs and outperformed other cybersecurity PLMs on most tasks. |