Papers by Simone Alghisi
DyKnow: Dynamically Verifying Time-Sensitive Factual Knowledge in LLMs (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Factual knowledge is subject to time-sensitive changes, and static benchmarks cannot address those cases. |
| Approach: | They propose to dynamically evaluate LLMs' knowledge and their time-sensitiveness against Wikidata, an up-to-date knowledge graph. |
| Outcome: | The proposed method compares LLMs and their time-sensitive knowledge against Wikidata, a publicly available up-to-date knowledge graph. |
CIVET: Systematic Evaluation of Understanding in VLMs (2025.findings-emnlp)
Copied to clipboard
Massimo Rizzoli, Simone Alghisi, Olha Khomyn, Gabriel Roccabruna, Seyed Mahed Mousavi, Giuseppe Riccardi
| Challenge: | Current Vision-Language Models can accurately recognize only a limited set of basic object properties; 3) they struggle to understand basic relations among objects. |
| Approach: | They propose a framework that evaluates VLMs on exhaustive sets of stimuli, free from annotation noise, dataset-specific biases, and uncontrolled scene complexity. |
| Outcome: | The proposed framework addresses the lack of standardized systematic evaluation for assessing VLMs’ understanding, enabling researchers to test hypotheses with statistical rigor. |