Papers by Gianluca Detommaso
Distance-aware Calibration for Pre-trained Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Language Models often produce overconfident predictions for both in-distribution and out-of-difference samples, i.e., the model’s output probabilities do not match their accuracy. |
| Approach: | They propose a post-hoc approach that changes the confidence scores of a Language Model by leveraging the distance between new samples and the in-domain training set. |
| Outcome: | The proposed approach improves in-domain calibration, robustness to different kind of distribution shift and also the model’s ability to detect out-of-distribution samples. |