Papers by Jianbing Shen
Visual In-Context Learning for Large Vision-Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to improve the performance of Large Visual Language Models (LVLMs) are limited by cross-modal interactions and representation disparities. |
| Approach: | They propose a Visual In-Context Learning method that retrieves images via a 'Retrieval & Rerank' paradigm and summarises images with task intent and task-specific visual parsing to compose language-based demonstrations that reduce token count. |
| Outcome: | The proposed method reduces token count and alleviates cross-modal interaction problem on visual reasoning datasets. |
Compatibility-Aware Dynamic Fine-Tuning for Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Recent work attributes optimization instability to the low probability of demonstrations being incompatible with the sample level. |
| Approach: | They propose a Dynamic Fine-Tuning extension of DFT that controls sample-level optimization variance. |
| Outcome: | The proposed model can generalize token-level stabilization to the sample level while remaining fully supervised and free of reward modeling. |
Self-Rewarding Large Vision-Language Models for Optimizing Prompts in Text-to-Image Generation (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for rewriting text-to-image models require specialized vocabulary . a new approach uses large vision language models to optimize text-based models . |
| Approach: | They propose a prompt optimization framework that rephrases a user prompt into a text-to-image model by using large vision language models as solver and reward model. |
| Outcome: | The proposed model outperforms existing models on two popular datasets. |
Improving Medical Large Vision-Language Models with Abnormal-Aware Feedback (2025.acl-long)
Copied to clipboard
| Challenge: | Existing Medical Large Vision-Language Models (Med-LVLMs) lack visual localization in medical images, which is crucial for abnormality detection and interpretation. |
| Approach: | They propose a medical abnormalities unveiling method based on a Medical Abnormalities Unveiler dataset and propose 'abnormal-aware instruction tuning' and 'abbnormal-Aware Reward' method generates diagnoses based upon identified abnormal areas in medical images. |
| Outcome: | The proposed method outperforms existing medical large vision-language models in identifying and understanding medical abnormalities and improves generalization capability. |
Multimodal Large Language Models for Multi-Subject In-Context Image Generation (2026.acl-long)
Copied to clipboard
| Challenge: | Recent advances in text-to-image (T2I) generation have enabled visually coherent image synthesis from descriptions, but generating images containing multiple given subjects remains challenging. |
| Approach: | They propose a model that enables automatic and scalable data generation without manual annotations to overcome the data scarcity. |
| Outcome: | The proposed model overcomes the data scarcity and lacks manual annotations. |
MAM: Modular Multi-Agent Framework for Multi-Modal Medical Diagnosis via Role-Specialized Collaboration (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in medical Large Language Models have demonstrated powerful reasoning and diagnostic capabilities. |
| Approach: | They propose a modular multi-agent framework for multi-modal medical diagnosis . they decompose the medical diagnostic process into specialized roles . |
| Outcome: | The framework decomposes the medical diagnostic process into specialized roles . it achieves significant performance improvements ranging from 18% to 365% compared to baseline models. |
Accelerating Training of Autoregressive Video Generation Models via Local Optimization with Representation Continuity (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing autoregressive models have shown superior performance and efficiency in image generation, but are constrained by high computational costs and prolonged training times in video generation. |
| Approach: | They propose a Local Optimization method which optimizes tokens within localized windows while leveraging contextual information to reduce error propagation. |
| Outcome: | The proposed method achieves superior performance to the baseline while halving the training cost without sacrificing quality. |