Papers by Xuming He
VLA-Mark: A cross modal watermark for large vision-language alignment models (2025.emnlp-main)
Copied to clipboard
Shuliang Liu, Zheng Qi, Jesse Jiaxi Xu, Yibo Yan, Junyan Zhang, He Geng, Aiwei Liu, Peijie Jiang, Jia Liu, Yik-Cheung Tam, Xuming Hu
| Challenge: | Existing text watermarking methods disrupt visual-textual alignment, leaving semantic-critical concepts vulnerable. |
| Approach: | They propose a vision-aligned framework that embeds detectable watermarks into outputs . they combine localized patch affinity, global semantic coherence, contextual attention patterns . |
| Outcome: | The proposed framework shows lower PPL and higher BLEU than conventional methods with near-perfect detection (98.8% AUC). |
Data Whisperer: Efficient Data Selection for Task-Specific LLM Fine-Tuning via Few-Shot In-Context Learning (2025.acl-long)
Copied to clipboard
Shaobo Wang, Xiangqi Jin, Ziming Wang, Jize Wang, Jiajun Zhang, Kaixin Li, Zichen Wen, Zhong Li, Conghui He, Xuming Hu, Linfeng Zhang
| Challenge: | Using fine-tuning on task-specific data is essential for large language models to be effective in specialized tasks. |
| Approach: | They propose a method that leverages few-shot in-context learning with the model to be fine-tuned. |
| Outcome: | The proposed method outperforms existing methods with a 3.1-point improvement and a 7.4 speedup on the Llama-3-8B-Instruct model using just 10% of the dataset. |
Are We Using the Right Benchmark: An Evaluation Framework for Visual Token Compression Methods (2026.acl-long)
Copied to clipboard
Chenfei Liao, Wensong Wang, Zichen Wen, Xu Zheng, Yiyu Wang, Haocong He, Yuanhuiyi Lyu, Lutao Jiang, Xin Zou, Yuqian Fu, Bin Ren, Linfeng Zhang, Xuming Hu
| Challenge: | Recent efforts to accelerate inference in Multimodal Large Language Models have focused on visual token compression. |
| Approach: | They propose a framework that leverages downsampling as a discriminator to denoise existing benchmarks. |
| Outcome: | The proposed evaluation framework leverages downsampling as a discriminator to denoise existing benchmarks. |
Comprehensive Benchmarking of Long-Form Speech Generation in Diverse Scenarios (2026.findings-acl)
Copied to clipboard
Changhao Pan, Rui Yang, Han Wang, Zhuan Zhou, Xuming He, Wenxiang Guo, Ziyue Jiang, Ruiqi Li, Yu Zhang, Chenyuhao Wen, Ke Lei, Xiang Yin, Jingyu Lu, Zhiyuan Zhu, Zhou Zhao
| Challenge: | Existing evaluation benchmarks for long-form speech are limited to limited domains, creating a significant gap with the diverse downstream applications. |
| Approach: | They propose a benchmark that decomposes "long-form speech quality" into specific, disentangled dimensions. |
| Outcome: | The proposed benchmark decomposes “long-form speech quality” into specific, disentangled dimensions. |
KD-VLP: Improving End-to-End Vision-and-Language Pretraining with Object Knowledge Distillation (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing vision-and-language pretraining approaches rely on external object detectors to encode images in a multi-modal transformer framework. |
| Approach: | They propose an object-aware end-to-end VLP framework which feeds image grid features from CNNs into the Transformer and learns the multi-modal representations jointly. |
| Outcome: | The proposed framework achieves competitive or superior performances on vision-language tasks. |
MarkLLM: An Open-Source Toolkit for LLM Watermarking (2024.emnlp-demo)
Copied to clipboard
Leyi Pan, Aiwei Liu, Zhiwei He, Zitian Gao, Xuandong Zhao, Yijian Lu, Binglin Zhou, Shuliang Liu, Xuming Hu, Lijie Wen, Irwin King, Philip Yu
| Challenge: | Large Language Models (LLMs) embed imperceptible yet algorithmically detectable signals in outputs to identify LLM-generated text. |
| Approach: | They propose to develop an open-source toolkit for LLM watermarking that embeds imperceptible yet algorithmically detectable signals in model outputs to identify LLM-generated text. |
| Outcome: | MarkLLM provides a unified framework for implementing LLM watermarking algorithms, while providing user-friendly interfaces to ensure ease of access. |
LLMArena: Assessing Capabilities of Large Language Models in Dynamic Multi-Agent Environments (2024.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks for evaluating large language models use static datasets, leading to data leakage or overlooking the complexities of multi-agent interactions. |
| Approach: | They propose a framework that evaluates the diverse capabilities of LLM agents in multi-agent dynamic environments. |
| Outcome: | The proposed framework assesses the diverse capabilities of LLM agents in multi-agent dynamic environments. |
A Survey on Proactive Defense Strategies Against Misinformation in Large Language Models (2025.findings-acl)
Copied to clipboard
Shuliang Liu, Hongyi Liu, Aiwei Liu, Duan Bingchen, Zheng Qi, Yibo Yan, He Geng, Peijie Jiang, Jia Liu, Xuming Hu
| Challenge: | Existing methods for detection of misinformation generated by large language models fail to mitigate societal risks . authors propose a paradigm shift from passive detection to anticipatory mitigation strategies . existing defenses remain reactionary in an era demanding proactive defense, authors say . |
| Approach: | They propose a three-pillar approach to prevent misinformation by fortifying integrity of training data and inference reliability by embedding self-corrective mechanisms during reasoning. |
| Outcome: | The proposed framework improves existing methods in misinformation prevention by 63% . it demonstrates that existing methods exhibit false negative rates against misinformation . |
A Survey of Mathematical Reasoning in the Era of Multimodal Large Language Model: Benchmark, Method & Challenges (2025.findings-acl)
Copied to clipboard
Yibo Yan, Jiamin Su, Jianxiang He, Fangteng Fu, Xu Zheng, Yuanhuiyi Lyu, Kun Wang, Shen Wang, Qingsong Wen, Xuming Hu
| Challenge: | This survey provides **the first comprehensive analysis of mathematical reasoning in the era of multimodal large language models** . integrating large language model with mathematical reasoning tasks is becoming significant as AI advances . |
| Approach: | They review over 200 studies published since 2021 and examine the state-of-the-art developments in Math-LLMs . they identify five major challenges hindering the realization of AGI in this domain . |
| Outcome: | The authors examine the state-of-the-art developments in Math-LLMs with a focus on multimodal settings. |