Papers by Lorenzo Torresani
UNICORN: A Unified Causal Video-Oriented Language-Modeling Framework for Temporal Video-Language Tasks (2024.emnlp-main)
Copied to clipboard
Yuanhao Xiong, Yixin Nie, Haotian Liu, Boxin Wang, Jun Chen, Rong Jin, Cho-Jui Hsieh, Lorenzo Torresani, Jie Lei
| Challenge: | Recent advances in large multimodal models have encouraged the development of large multi-modal models . however, it is unclear how to extend these models to the more complex video domain . |
| Approach: | They propose a visual instruction tuning framework to address temporal video-language tasks . they collect a dataset and fine-tune the framework on instruction-following data . |
| Outcome: | The proposed model can perform better on established temporal video-language tasks without training objectives and intensive pre-training. |