Papers by Zejun Li
Negative Sample is Negative in Its Own Way: Tailoring Negative Sentences for Image-Text Retrieval (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing approaches to retrieve hard negative sentences are limited in the scale of the dataset thus fail to identify negative samples of high difficulty for every image. |
| Approach: | They propose to use a model to generate synthetic negative sentences with higher difficulty by masking and refilling the images and performing word discrimination and word correction tasks to improve retrieval and generation. |
| Outcome: | The proposed model generates synthetic negative sentences with higher difficulty on MS-COCO and Flickr30K and is robust and faithful to state-of-the-art training. |
VoCoT: Unleashing Visually Grounded Multi-Step Reasoning in Large Multi-Modal Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Despite the impressive capabilities of large multi-modal models, their effectiveness in handling complex tasks has been limited by the prevailing singlestep reasoning paradigm. |
| Approach: | They propose a visuallygrounded object-centric Chain-of-Thought reasoning framework for LMMs that is based on a multi-modal interleaved and aligned representation of object concepts. |
| Outcome: | The proposed model outperforms SOTA models in CLEVR and EmbSpatial benchmarks. |
Ready Jurist One: Benchmarking Language Agents for Legal Intelligence in Dynamic Environments (2026.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks for legal intelligence are limited to static evaluation paradigms or simplified scenarios. |
| Approach: | They introduce J1-ENVS, the first interactive and dynamic legal environment tailored for LLM-based agents. |
| Outcome: | The proposed framework assesses task performance and procedural compliance across legal proficiency levels. |
Simple-VGC: Enhancing Visual Grounding in Multimodal Reasoning via Adaptive Tool Composition (2026.acl-long)
Copied to clipboard
| Challenge: | Existing multimodal large language models suffer from systematic failures in basic visual understanding. |
| Approach: | They propose a tool-augmented reasoning framework with three targeted compensation strategies to address these problems. |
| Outcome: | The proposed framework improves visual grounding by re-injecting the original image to mitigate visual forgetting, the authors show . the proposed framework also improves the accuracy of the visual inputs, the researchers show - and the results are promising . |
AutoJudger: An Agent-Driven Framework for Efficient Benchmarking of MLLMs (2026.acl-long)
Copied to clipboard
| Challenge: | Evaluating multimodal large language models (MLLMs) is becoming increasingly expensive as benchmarks grow in scale and cross-modality complexity. |
| Approach: | They propose an adaptive evaluation framework for efficient benchmarking that treats evaluation as an interview-like process by keeping a hypothesized ability structure of the evaluated model and actively selecting the informative questions. |
| Outcome: | Experiments on four representative multimodal benchmarks show that **A2-Judger significantly improves sample efficiency while maintaining reliable evaluation results. |
MMSearch-R1: Incentivizing LMMs to Search (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to deploying large multimodal models rely on rigid pipelines . Existing methods such as retrieval-augmented generation and prompt engineered search rely only on rigid knowledge sources. |
| Approach: | They propose a framework that enables LMMs to perform on-demand, multi-turn search in real-world Internet environments. |
| Outcome: | The proposed model outperforms existing models while reducing search calls by over 30%. |
Audio-centric Video Understanding Benchmark without Text Shortcut (2025.emnlp-main)
Copied to clipboard
Yudong Yang, Jimin Zhuang, Guangzhi Sun, Changli Tang, Yixuan Li, Peihan Li, Yifan Jiang, Wei Li, Zejun Ma, Chao Zhang
| Challenge: | Recent advances in multimodal large language models (MLLMs) focus on visual abilities, but audio is essential for video understanding. |
| Approach: | They propose an audio-centric video understanding benchmark to evaluate video comprehension capabilities of multimodal LLMs with a particular focus on auditory information. |
| Outcome: | The proposed video understanding benchmarks evaluate video comprehension capabilities of multimodal models with a particular focus on auditory information. |
Activating Distributed Visual Region within LLMs for Efficient and Effective Vision-Language Training and Inference (2025.acl-long)
Copied to clipboard
| Challenge: | Existing Large Vision-Language Models (LVLMs) learn visual capacity through visual instruction tuning. |
| Approach: | They propose a method for LVLMs to be trained by selective layers tuning . they propose removing non-critical layers outside the visual region . |
| Outcome: | The proposed approach preserves nearly 99% of visual performance and improves textual task results while reducing training time. |
DELAN: Dual-Level Alignment for Vision-and-Language Navigation by Cross-Modal Contrastive Learning (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing studies focus on cross-modal attention at the fusion stage, but modality features generated by disparate uni-encoders reside in their own spaces, leading to a decline in the quality of cross-modulation and decision-making. |
| Approach: | They propose a framework to align navigation-related modalities before fusion by cross-modal contrastive learning. |
| Outcome: | The proposed framework integrates with the majority of existing models, resulting in improved navigation performance on various VLN benchmarks, including R2R, R4R, and CVDN. |
Unifying Cross-Lingual and Cross-Modal Modeling Towards Weakly Supervised Multilingual Vision-Language Pre-training (2023.acl-long)
Copied to clipboard
| Challenge: | Existing studies address the problem of translating English data into other languages, but they are limited in form and scale. |
| Approach: | They propose a framework to unify cross-lingual and cross-modal pre-training by using English data. |
| Outcome: | The proposed framework unifies cross-lingual and cross-modal pre-training on different data. |
EmbSpatial-Bench: Benchmarking Spatial Understanding for Embodied Tasks with Large Vision-Language Models (2024.acl-short)
Copied to clipboard
| Challenge: | Recent studies have revealed significant deficiencies of LVLMs in understanding visual contents, leaving the gap between current embodied intelligence and large vision-language models (LVLM) . |
| Approach: | They propose to use a benchmark to evaluate LVLMs' spatial understanding of embodied environments to evaluate their ability to understand visual contents. |
| Outcome: | The proposed benchmark is derived from embodied scenes and covers 6 spatial relationships from an egocentric perspective. |