Papers by Yi-Lin Sung
An Empirical Study of Multimodal Model Merging (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies have shown that model merging can generate a multi-task solution without synchronous training. |
| Approach: | They propose to merge vision, language, and cross-modal transformers of a modality-specific architecture to create a parameter-efficient architecture. |
| Outcome: | The proposed model merging outperforms naive models on various tasks with improvements of 3% on VQA, 7% on COCO retrieval, 25% on NLVR2, 14% on Flickr30k and 3% ADE20k. |
Glider: Global and Local Instruction-Driven Expert Router (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for routing-based expert models favor generalization over performance on held-in tasks. |
| Approach: | They propose a global and local instruction driven expert router that leverages recent LLMs' semantic reasoning capabilities to generate task-specific instructions from the input query. |
| Outcome: | The proposed method improves held-in performance while maintaining strong generalization on held-out tasks. |