Papers by Zhuowan Li
Localization vs. Semantics: Visual Representations in Unimodal and Multimodal Models (2024.eacl-long)
Copied to clipboard
| Challenge: | Existing vision-and-language models perform better on multimodal tasks, but there is little understanding of how multimodal learning can help visual representations. |
| Approach: | They conduct a probing analysis of visual representations in existing vision-and-language models and vision-only models by probing on a broad range of tasks. |
| Outcome: | The proposed model improves vision-and-language models on label and attribute prediction tasks while vision-only models are stronger on dense prediction tasks. |
Visual Commonsense in Pretrained Unimodal and Multimodal Models (2022.naacl-main)
Copied to clipboard
| Challenge: | Fig. 1 shows how text-only and image-only models can capture commonsense visual attributes, but reporting bias affects their performance. |
| Approach: | They use a Visual Commonsense Tests dataset to validate their findings . they find multimodal models better reconstruct attribute distributions, but are still subject to reporting bias . |
| Outcome: | The proposed model improves on the unimodal and multimodal models, but is still subject to reporting bias. |
Retrieval Augmented Generation or Long-Context LLMs? A Comprehensive Study and Hybrid Approach (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Recent LLMs like Gemini-1.5 and GPT-4 show exceptional capabilities to understand long contexts directly. |
| Approach: | They propose a method that routes queries to RAG or LC based on model self-reflection. |
| Outcome: | The proposed method significantly reduces the computation cost while maintaining a comparable performance to RAG. |