Papers with POE
Pseudo Outlier Exposure for Out-of-Distribution Detection using Pretrained Transformers (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to detect out-of-distribution (OOD) samples are overconfident for real-world language applications. |
| Approach: | They propose a method that constructs a surrogate OOD dataset by sequentially masking tokens related to ID classes. |
| Outcome: | The proposed method can train a rejection network with ID and diverse outlier samples but requires additional data collection overhead. |
POE: Process of Elimination for Multiple Choice Reasoning (2023.emnlp-main)
Copied to clipboard
| Challenge: | Current language models perform well on multiple choice reasoning tasks, but the options are not treated equally. |
| Approach: | They propose a two-step scoring method that scores options and masks them to make the final prediction from the remaining options. |
| Outcome: | The proposed method is especially performant on logical reasoning tasks. |