Papers by Kankan Zhou
VLStereoSet: A Study of Stereotypical Bias in Pre-trained Vision-Language Models (2022.aacl-main)
Copied to clipboard
| Challenge: | Existing studies on pre-trained vision-language models have focused on measuring biases and stereotypes in a single modality. |
| Approach: | They extend a recently released stereotypical bias dataset into a vision-language probing dataset called VLStereoSet to measure stereotypical biased vision-linguistic models. |
| Outcome: | The proposed probing task measures stereotypical bias in vision-language models and its intra-modal and inter-modal biases. |
ROME: Evaluating Pre-trained Vision-Language Models on Reasoning beyond Visual Common Sense (2023.findings-emnlp)
Copied to clipboard
| Challenge: | a vision-language model with commonsense knowledge can reason beyond common sense . however, pre-trained vision-linguistic models are incapable of interpreting counter-intuitive content . |
| Approach: | They introduce a probing dataset to evaluate vision-language models' reasoning abilities . they use images that defy commonsense knowledge to test their reasoning abilities. |
| Outcome: | The proposed dataset evaluates whether pre-trained vision-language models can reason beyond common sense . it contains images that defy commonsense knowledge with regards to color, shape, material, size and position . |
FOCUS: Evaluating Pre-trained Vision-Language Models on Underspecification Reasoning (2025.acl-long)
Copied to clipboard
| Challenge: | a new dataset evaluates whether vision-language models have underspecification reasoning abilities . underspecifications are often left incomplete or vague, and are often ignored for mutual understanding . |
| Approach: | They propose a probing dataset to evaluate whether VLMs have underspecification reasoning . they find that pre-trained vision-language models lack this ability . |
| Outcome: | The proposed probing dataset shows that pre-trained vision-language models lack underspecification reasoning abilities. |