Papers by Zhien Zhang
Not quite Sherlock Holmes: Language model predictions do not reliably differentiate impossible from improbable events (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing work has shown that language models can select the most likely or plausible of a set of possible events, but they are far from robust. |
| Approach: | They focus on whether language models can select the most likely or plausible of a set of possibilities and compare them to a broader behavior that humans exhibit largely unconsciously. |
| Outcome: | The proposed models perform worse than expected under certain conditions, compared with Llama 3, Gemma 2, and Mistral NeMo, and they are significantly more sensible than leaves. |