Papers by Robert Wolfe
Low Frequency Names Exhibit Bias and Overfitting in Contextualizing Language Models (2021.emnlp-main)
Copied to clipboard
| Challenge: | Infrequent names are less similar to initial representations, and are more self-similar, suggesting that models rely on less context-informed representations of uncommon and minority names. |
| Approach: | They use a dataset of U.S. first names with labels based on predominant gender and racial group to examine effect of training corpus frequency on tokenization, contextualization, similarity to initial representation, and bias. |
| Outcome: | The results show that infrequent names are less similar to initial representations and have a Spearman’s rho between frequency and self-similarity as low as .763 . |
Contrastive Visual Semantic Pretraining Magnifies the Semantics of Natural Language Representations (2022.acl-long)
Copied to clipboard
| Challenge: | Large-scale "natural language supervision" using image captions has enabled the first "zero-shot" AI image classifiers, which allow users to create their own image classes using natural language, yet outperform supervised models on common language-and-image tasks. |
| Approach: | They compare the geometry and semantic properties of contextualized English language representations formed by GPT-2 and CLIP, a zero-shot multimodal image classifier which adapts the GPT2 architecture to encode image captions. |
| Outcome: | The proposed classifier outperforms GPT-2 on word-level semantic intrinsic evaluation tasks and achieves a new corpus-based state of the art for the RG65 evaluation. |
Do Language Models Mirror Human Confidence? Exploring Psychological Insights to Address Overconfidence in LLMs (2025.findings-acl)
Copied to clipboard
| Challenge: | Psychology research has shown that humans are poor at estimating their performance on tasks, tending towards underconfidence on easy tasks and overconfidence on difficult tasks. |
| Approach: | They propose to use a self-assessment method to assess confidence in large language models (LLMs) they propose to ask for the answer separately and then use them to improve their accuracy. |
| Outcome: | The proposed method improves confidence calibration and interpretability in QA tasks with different personas. |