Papers by Yucheng Zhou
Visual In-Context Learning for Large Vision-Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to improve the performance of Large Visual Language Models (LVLMs) are limited by cross-modal interactions and representation disparities. |
| Approach: | They propose a Visual In-Context Learning method that retrieves images via a 'Retrieval & Rerank' paradigm and summarises images with task intent and task-specific visual parsing to compose language-based demonstrations that reduce token count. |
| Outcome: | The proposed method reduces token count and alleviates cross-modal interaction problem on visual reasoning datasets. |
Improving Cross-modal Alignment for Text-Guided Image Inpainting (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing methods allocate most of computation to visual encoding, while light computation on modeling modality interactions. |
| Approach: | They propose a novel model for text-guided image inpainting by improving cross-modal alignment knowledge by using a vision-language encoder and an image generator. |
| Outcome: | The proposed model achieves state-of-the-art performance compared with other strong competitors on two vision-language datasets. |
Multimodal Event Transformer for Image-guided Story Ending Generation (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing methods focus on cross-modal feature fusion but overlook reasoning and mining implicit information from story plots and ending image. |
| Approach: | They propose a multimodal event transformer framework for image-guided story ending generation. |
| Outcome: | The proposed method achieves state-of-the-art performance for image-guided story ending generation. |
Compatibility-Aware Dynamic Fine-Tuning for Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Recent work attributes optimization instability to the low probability of demonstrations being incompatible with the sample level. |
| Approach: | They propose a Dynamic Fine-Tuning extension of DFT that controls sample-level optimization variance. |
| Outcome: | The proposed model can generalize token-level stabilization to the sample level while remaining fully supervised and free of reward modeling. |
ORANGE: Text-video Retrieval via Watch-time-aware Heterogeneous Graph Contrastive Learning (2023.emnlp-industry)
Copied to clipboard
Yucheng Lin, Tim Chang, Yaning Chang, Jianqiang Ma, Donghui Li, Ting Peng, Zang Li, Zhiyi Zhou, Feng Wang
| Challenge: | Existing methods for text-video retrieval focus on informative representations and delicate matching mechanisms, but real-world scenarios often involve brief, ambiguous queries and low-quality videos. |
| Approach: | They propose a novel method to learn informative embeddings for queries and videos . they use a watch-time-aware contrastive learning paradigm to capture dependencies . |
| Outcome: | The proposed method is effective in a real-world video-search service. |
ClarET: Pre-training a Correlation-Aware Context-To-Event Transformer for Event-Centric Generation and Classification (2022.acl-long)
Copied to clipboard
| Challenge: | Existing work on event-centric reasoning fails to model event-level correlations . Existing studies limit their scope to specific scenarios or overlook event- level correlations. |
| Approach: | They propose to pre-train a general Correlation-aware context-to-Event Transformer for event-centric reasoning by highlighting event-level correlations with effective training. |
| Outcome: | The proposed model is applicable to a wide range of event-centric reasoning scenarios, considering its versatility of event correlation types, application formulations, and reasoning types. |
LADR: Locality-Aware Dynamic Rescue for Efficient Text-to-Image Generation with Diffusion Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for inference are expensive and lack spatial redundancy . Discrete Diffusion Language Models are a promising paradigm for multimodal generation . |
| Approach: | They propose a locality-aware dynamic rescue method that exploits spatial Markov property of images. |
| Outcome: | The proposed method achieves an approximate 4 speedup over baselines on four text-to-image generation benchmarks. |
Self-Rewarding Large Vision-Language Models for Optimizing Prompts in Text-to-Image Generation (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for rewriting text-to-image models require specialized vocabulary . a new approach uses large vision language models to optimize text-based models . |
| Approach: | They propose a prompt optimization framework that rephrases a user prompt into a text-to-image model by using large vision language models as solver and reward model. |
| Outcome: | The proposed model outperforms existing models on two popular datasets. |
NoteChat: A Dataset of Synthetic Patient-Physician Conversations Conditioned on Clinical Notes (2024.findings-acl)
Copied to clipboard
| Challenge: | NoteChat is a cooperative multi-agent framework for generating patient-physician dialogues . evaluator finds it outperforms state-of-the-art models for generating clinical notes . clinical documentation is largely done by physicians at both steps . |
| Approach: | They propose a cooperative multi-agent framework leveraging Large Language Models to generate patient-physician dialogues. |
| Outcome: | The proposed framework outperforms state-of-the-art models for generating clinical notes . it can engage patients directly and help clinical documentation, a leading cause of physician burnout . |
LLM Inductive Reasoning Through Multi-Agent Enhanced Monte Carlo Tree Search (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for enhancing inductive reasoning of large language models often lack explicit optimization guidance and effective error correction. |
| Approach: | They propose a plug-and-play test-time framework that integrates multi-agent coordination with Monte Carlo Tree Search to improve inductive reasoning. |
| Outcome: | The proposed framework outperforms existing methods on four benchmarks and shows consistent improvements on QWQ-32B and Deepseek-V3 . |
Modeling Event-Pair Relations in External Knowledge Graphs for Script Reasoning (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing methods focus on graph triples with event overlap, but ignore more supportive triples . Script reasoning relies on understanding the relationship between two events . |
| Approach: | They propose a model to learn the inferential relations between events from the whole eventuality KG . they propose 'script adapter' to extend the model to infer the associated relations between an event chain and a subsequent event candidate. |
| Outcome: | The proposed model is compared with baselines using external KG or not on a script reasoning task. |
Style-Aware Contrastive Learning for Multi-Style Image Captioning (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing multi-style image captioning methods focus on visual content and style . existing methods overlook the relationship between linguistic style and visual content. |
| Approach: | They propose a style-aware visual encoder with contrastive learning to mine potential visual content relevant to style and a triplet contrast objective to distinguish whether the image, style and caption matched. |
| Outcome: | The proposed method achieves state-of-the-art performance and an extensive analysis to verify its effectiveness. |
Improving Medical Large Vision-Language Models with Abnormal-Aware Feedback (2025.acl-long)
Copied to clipboard
| Challenge: | Existing Medical Large Vision-Language Models (Med-LVLMs) lack visual localization in medical images, which is crucial for abnormality detection and interpretation. |
| Approach: | They propose a medical abnormalities unveiling method based on a Medical Abnormalities Unveiler dataset and propose 'abnormal-aware instruction tuning' and 'abbnormal-Aware Reward' method generates diagnoses based upon identified abnormal areas in medical images. |
| Outcome: | The proposed method outperforms existing medical large vision-language models in identifying and understanding medical abnormalities and improves generalization capability. |
Impromptu Cybercrime Euphemism Detection (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods for detecting euphemisms are ineffective in impromptu euphorism detection . Existing approaches for e-mail detection are limited to word-level ephemismals . |
| Approach: | They propose a framework for impromptu euphemism detection that integrates context augmentation and multi-round iterative training to better predict the actual meaning of a masked token. |
| Outcome: | The proposed framework improves 76-fold over the previous state-of-the-art euphemism detector. |
SURf: Teaching Large Vision-Language Models to Selectively Utilize Retrieved Information (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies focus on the text modality or are limited to specific tasks. |
| Approach: | They propose a framework to teach Large Vision-Language Models to selectively utilize retrieved information and improve their robustness against irrelevant or misleading references. |
| Outcome: | The proposed framework improves LVLMs’ ability to utilize retrieved multimodal references and their robustness against irrelevant or misleading information. |
Improving Zero-Shot Cross-lingual Transfer for Multilingual Question Answering over Knowledge Graph (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to solve multilingual question answering over knowledge graph (KGQA) use of bilingual lexicon induction to map training questions into those in target language circumvents language inconsistency . |
| Approach: | They propose to use bilingual lexicon induction to map training questions in source and target languages as augmented training data to minimize syntax-disorder. |
| Outcome: | The proposed model narrows the gap in zero-shot cross-lingual transfer between source and target languages. |
Towards Robust Ranker for Text Retrieval (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for text retrieval are based on a 'retrieval & rerank' pipeline, which uses a fast retriever to fetch a set of top document candidates, while a robust ranker is based upon a weak negative mining during contrastive learning. |
| Approach: | They propose a multi-adversarial training strategy that leverages multiple retrievers as generators to challenge a ranker. |
| Outcome: | The proposed model outperforms the existing de facto ranker training paradigms on the passage retrieval benchmarks using BM25-reranking, full-ranking and retriever distillation. |
Multimodal Large Language Models for Multi-Subject In-Context Image Generation (2026.acl-long)
Copied to clipboard
| Challenge: | Recent advances in text-to-image (T2I) generation have enabled visually coherent image synthesis from descriptions, but generating images containing multiple given subjects remains challenging. |
| Approach: | They propose a model that enables automatic and scalable data generation without manual annotations to overcome the data scarcity. |
| Outcome: | The proposed model overcomes the data scarcity and lacks manual annotations. |
ASDOT: Any-Shot Data-to-Text Generation with Pretrained Language Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to data-to-text generation require limited training examples . a data-based approach is based on a set of pre-trained language models with optional finetuning. |
| Approach: | They propose a data-to-text generation task that makes use of any given (or no) examples. |
| Outcome: | The proposed approach improves on baselines on a dataset with zero/few/full-shot settings. |
Automatic Essay Scoring Incorporating Rating Schema via Reinforcement Learning (D18-1)
Copied to clipboard
| Challenge: | Existing systems for automatic essay scoring are trained to predict the score of each essay at a time without considering rating schema. |
| Approach: | They propose a reinforcement learning framework that incorporates quadratic weighted kappa as guidance to optimize the scoring system. |
| Outcome: | Experiments on benchmark datasets show the proposed framework is effective. |
Safety Alignment via Constrained Knowledge Unlearning (2025.acl-long)
Copied to clipboard
Zesheng Shi, Yucheng Zhou, Jing Li, Yuxin Jin, Yu Li, Daojing He, Fangming Liu, Saleh Alharbi, Jun Yu, Min Zhang
| Challenge: | Existing defense mechanisms have not fully deleted harmful knowledge in large language models (LLMs) Existing methods to address safety alignment have not completely deleted harmful information in LLMs. |
| Approach: | They propose a safety alignment strategy that uses scoring neurons to identify useful knowledge in LLMs and pruning the gradients of neurons in U to preserve beneficial information. |
| Outcome: | The proposed method significantly improves model safety while maintaining utility compared to existing methods. |
MAM: Modular Multi-Agent Framework for Multi-Modal Medical Diagnosis via Role-Specialized Collaboration (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in medical Large Language Models have demonstrated powerful reasoning and diagnostic capabilities. |
| Approach: | They propose a modular multi-agent framework for multi-modal medical diagnosis . they decompose the medical diagnostic process into specialized roles . |
| Outcome: | The framework decomposes the medical diagnostic process into specialized roles . it achieves significant performance improvements ranging from 18% to 365% compared to baseline models. |
Accelerating Training of Autoregressive Video Generation Models via Local Optimization with Representation Continuity (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing autoregressive models have shown superior performance and efficiency in image generation, but are constrained by high computational costs and prolonged training times in video generation. |
| Approach: | They propose a Local Optimization method which optimizes tokens within localized windows while leveraging contextual information to reduce error propagation. |
| Outcome: | The proposed method achieves superior performance to the baseline while halving the training cost without sacrificing quality. |