Papers by Alexander Huth
Selecting Informative Contexts Improves Language Model Fine-tuning (2021.acl-long)
Copied to clipboard
| Challenge: | Language model fine-tuning is computationally expensive and time-consuming . however, the inclusion of training examples that negatively affect performance is limited . |
| Approach: | They propose a general fine-tuning method that incorporates information gain filtration . they propose to release pre-trained secondary learners on common corpora to promote efficient fine-uning. |
| Outcome: | The proposed method achieves a median perplexity of 54.0 on a books dataset compared to 57.3 for standard fine-tuning. |