Papers by Lingkai Kong
Calibrated Language Model Fine-Tuning for In- and Out-of-Distribution Data (2020.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained language models suffer from severe miscalibration for both in-distribution and out-of-difference data due to over-parameterization. |
| Approach: | They propose a regularized method to improve in-distribution and out-of-distance calibrations by using on-manifold regularization and off-manfold regularisation. |
| Outcome: | The proposed method outperforms existing methods for text classification in terms of expectation calibration error, misclassification detection, and OOD detection on six datasets. |
AcTune: Uncertainty-Based Active Self-Training for Active Fine-Tuning of Pretrained Language Models (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for fine-tuning pre-trained language models ignore the potential of unlabeled data. |
| Approach: | They propose a framework that allows users to unleash the power of unlabeled data via self-training. |
| Outcome: | The proposed framework outperforms active learning and self-training baselines and improves the label efficiency of PLM fine-tuning by 56.2% on average. |