Papers by Bill Howe
Do Language Models Mirror Human Confidence? Exploring Psychological Insights to Address Overconfidence in LLMs (2025.findings-acl)
Copied to clipboard
| Challenge: | Psychology research has shown that humans are poor at estimating their performance on tasks, tending towards underconfidence on easy tasks and overconfidence on difficult tasks. |
| Approach: | They propose to use a self-assessment method to assess confidence in large language models (LLMs) they propose to ask for the answer separately and then use them to improve their accuracy. |
| Outcome: | The proposed method improves confidence calibration and interpretability in QA tasks with different personas. |
Characterizing LLM Abstention Behavior in Science QA with Context Perturbations (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Prior work has investigated the ability of LLMs to abstain from answering context-dependent questions when provided insufficient or inconsistent context is provided. |
| Approach: | They propose to improve abstention when provided insufficient or incorrect context . they probed the ability of LLMs to abstain from answering context-dependent science questions . |
| Outcome: | The proposed models abstain from answering science questions when provided insufficient or incorrect context. |