Papers by Radu Timofte
Babel-ImageNet: Massively Multilingual Evaluation of Vision-and-Language Representations (2024.acl-long)
Copied to clipboard
| Challenge: | Vision-and-language models with separate encoders for each modality are limited in availability. |
| Approach: | They propose a multilingual benchmark that offers (partial) translations of ImageNet labels to 100 languages, built without machine translation or manual annotation. |
| Outcome: | The proposed model outperforms models on English and low-resource languages. |
e5-omni: Explicit Cross-modal Alignment for Omni-modal Embeddings (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent omni-modal embeddings rely heavily on implicit alignment from pretrained visionlanguage models. |
| Approach: | They propose a lightweight explicit alignment recipe that adapts off-the-shelf VLMs into robust omni-modal embedding models. |
| Outcome: | The proposed model improves on MMEB-V2 and AudioCaps with a lightweight explicit alignment recipe. |
African or European Swallow? Benchmarking Large Vision-Language Models for Fine-Grained Object Classification (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent Large Vision Language Models demonstrate impressive abilities on image understanding and reasoning tasks. |
| Approach: | They propose a benchmark for fine-grained object classification that is difficult to evaluate . they benchmark 12 public LVLMs on and show CLIP models exhibit better performance . |
| Outcome: | The proposed model improves on 12 public LVLMs on image understanding and reasoning tasks. |
Does Object Grounding Really Reduce Hallucination of Large Vision-Language Models? (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large vision-language models (LVLMs) often hallucinate and produce captions that mention concepts that cannot be found in the image. |
| Approach: | They propose to add grounding objectives to captions that explicitly align image regions or objects to text spans to reduce hallucination. |
| Outcome: | The proposed evaluation protocol reduces the amount of hallucination in LVLMs by adding grounding objectives. |
Centurio: On Drivers of Multilingual Ability of Large Vision-Language Model (2025.acl-long)
Copied to clipboard
Gregor Geigle, Florian Schneider, Carolin Holtermann, Chris Biemann, Radu Timofte, Anne Lauscher, Goran Glavaš
| Challenge: | Existing models for large vision-language tasks are trained on English data, which makes them struggle to understand non-English input and fail to generate output in the desired target language. |
| Approach: | They conduct multi-stage experiments on 13 vision-language tasks and 43 languages . they find that one can include as many as 100 training languages simultaneously with as little as 25-50% of non-English data . |
| Outcome: | The proposed model outperforms existing models in 14 tasks and 56 languages. |