Papers by Jinfa Huang
Evolver: Chain-of-Evolution Prompting to Boost Large Multimodal Models for Hateful Meme Detection (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods for detecting hateful memes rely on extensive training. |
| Approach: | They propose a method that integrates evolution attribute and in-context information of memes into large multimodal models via Chain-of-Evolution (CoE) prompting. |
| Outcome: | The proposed method improves existing methods on public datasets and can be used as interpretive tool to promote understanding of evolution of memes. |
TACO: Enhancing Multimodal In-context Learning via Task Mapping-Guided Sequence Configuration (2025.emnlp-main)
Copied to clipboard
| Challenge: | Multimodal in-context learning (ICL) is a key mechanism for harnessing the capabilities of large vision–language models. |
| Approach: | They propose a transformer-based model with task-aware attention that dynamically configures ICL sequences. |
| Outcome: | Experiments on five LVLMs and nine datasets show that TACO surpasses baselines across diverse ICL tasks. |
LOOK-M: Look-Once Optimization in KV Cache for Efficient Multimodal Long-Context Inference (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Long-context Multimodal Large Language Models (MLLMs) require substantial computational resources for inference . the growth of their multimodal Key-Value (KV) cache challenges memory and time efficiency. |
| Approach: | They propose a fine-tuning-free approach that efficiently reduces the multimodal KV cache size while maintaining performance comparable to a full cache. |
| Outcome: | The proposed method reduces the multimodal KV cache size while maintaining performance comparable to a full cache. |
RAP: Efficient Text-Video Retrieval with Sparse-and-Correlated Adapter (2024.findings-acl)
Copied to clipboard
Meng Cao, Haoran Tang, Jinfa Huang, Peng Jin, Can Zhang, Ruyang Liu, Long Chen, Xiaodan Liang, Li Yuan, Ge Li
| Challenge: | Text-Video Retrieval (TVR) aims to align relevant video content with natural language queries. |
| Approach: | They propose to conduct efficient text-video Retrieval with a salient-and-correlated AdaPter . they propose a low-rank modulation module to refine per-image features from frozen CLIP backbone . |
| Outcome: | Experiments on four TVR datasets show that the proposed method performs better than other methods. |