Papers by Jialin Wu
CoNAN: A Complementary Neighboring-based Attention Network for Referring Expression Generation (2020.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to identify and describe complex daily scenes are limited by ambiguity. |
| Approach: | They propose a Complementary Neighboring-based Attention Network that utilizes visual differences between the target object and its highly-related neighbors as complementary features. |
| Outcome: | The proposed expression outperforms state-of-the-art models on a dataset of 3D objects. |
Convolutional LoRA Aggregation for Unseen Tasks Adaptation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing LoRA selection methods rely on a few task samples, making it difficult to capture full scope of task-relevant information. |
| Approach: | They propose a framework that selects appropriate LoRA modules and aggregates them using a convolutional LoRA aggregator. |
| Outcome: | The proposed framework bridges the knowledge gap between selected modules and target task . it ensures comprehensive coverage of task-relevant LoRA modules . |
From Behavioral Performance to Internal Competence: Interpreting Vision-Language Models with VLM-Lens (2025.emnlp-demos)
Copied to clipboard
Hala Sheta, Eric Haoran Huang, Shuyu Wu, Ilia Alenabi, Jiajun Hong, Ryker Lin, Ruoxi Ning, Daniel Wei, Jialin Yang, Jiawei Zhou, Ziqiao Ma, Freda Shi
| Challenge: | Existing vision-language models are based on exactmatch based accuracy and its derivations to evaluate performance. |
| Approach: | They propose a toolkit that supports systematic benchmarking, analysis, and interpretation of vision-language models by extracting intermediate outputs from any layer during the forward pass of open-source VLMs. |
| Outcome: | The proposed toolkit supports 16 state-of-the-art base VLMs and their over 30 variants and is extensible to accommodate new models without changing the core logic. |
Entity-Focused Dense Passage Retrieval for Outside-Knowledge Visual Question Answering (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing outsideknowledge visual question answering systems lack retrieved knowledge and supervision is weak . |
| Approach: | They propose an Entity-Focused Retrieval model that provides stronger supervision during training and recognizes question-relevant entities to help retrieve more specific knowledge. |
| Outcome: | The proposed model achieves superior retrieval performance on the currently largest outside-knowledge VQA dataset. |
Generating Question Relevant Captions to Aid Visual Question Answering (P19-1)
Copied to clipboard
| Challenge: | Visual question answering and image captioning require a shared body of general knowledge connecting language and vision. |
| Approach: | They propose a method that exploits a shared body of general knowledge connecting language and vision by jointly generating captions. |
| Outcome: | The proposed approach obtains state-of-the-art performance on the VQA v2 challenge . it uses human annotated captions to generate question-relevant captions . |