Papers by Junyan Lin
Multimodal Language Models See Better When They Look Shallower (2025.emnlp-main)
Copied to clipboard
Haoran Chen, Junyan Lin, Xinghao Chen, Yue Fan, Jianfeng Dong, Xin Jin, Hui Su, Jinlan Fu, Xiaoyu Shen
| Challenge: | Existing studies show that multimodal large language models extract visual features from the final layers of a pretrained Vision Transformer. |
| Approach: | They propose a feature fusion method that strategically incorporates shallower layers . they propose MLLMs that extract visual features from the final layers of a pretrained Vision Transformer . |
| Outcome: | The proposed method outperforms deep layers on fine-grained visual tasks . it is the first comprehensive study of visual layer selection for MLLMs . |
To Preserve or To Compress: An In-Depth Study of Connector Selection in Multimodal Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent multimodal large language models (MLLMs) have attracted widespread attention from both industry and academia due to their potential to handle multiple modalities in a unified framework. |
| Approach: | They propose to classify connectors into feature-preserving and feature-compressing types and categorize tasks into three task types: coarse-grained perception, fine-grain perception, and reasoning. |
| Outcome: | The proposed architectures perform better on tasks with varying granularities than on external fusion architectures. |