Papers by Xuming Hu

VLA-Mark: A cross modal watermark for large vision-language alignment models (2025.emnlp-main)

Challenge:	Current scientific reasoning models struggle with generalization across domains and fall short of multimodal perception.
Approach:	They propose to use multimodal large language models to integrate text, images, and other modalities to enhance scientific reasoning.
Outcome:	The proposed models can integrate text, images, and other modalities and improve reasoning across disciplines.

Copied to clipboard

Shuliang Liu, Zheng Qi, Jesse Jiaxi Xu, Yibo Yan, Junyan Zhang, He Geng, Aiwei Liu, Peijie Jiang, Jia Liu, Yik-Cheung Tam, Xuming Hu

Robustness in NLP

Challenge:	Existing text watermarking methods disrupt visual-textual alignment, leaving semantic-critical concepts vulnerable.
Approach:	They propose a vision-aligned framework that embeds detectable watermarks into outputs . they combine localized patch affinity, global semantic coherence, contextual attention patterns .
Outcome:	The proposed framework shows lower PPL and higher BLEU than conventional methods with near-perfect detection (98.8% AUC).

SelfORE: Self-supervised Relational Feature Learning for Open Relation Extraction (2020.emnlp-main)

Copied to clipboard

Xuming Hu, Lijie Wen, Yusong Xu, Chenwei Zhang, Philip Yu

PhysicsArena: The First Multimodal Physics Reasoning Benchmark Exploring Variable, Process, and Solution Dimensions (2025.findings-emnlp)

Challenge:	Existing methods for relation extraction use heuristics or distant-supervised annotations, but distant supervised methods make strong assumptions on entity cooccurrence without sufficient contexts.
Approach:	They propose a framework that exploits weak, self-supervised signals by leveraging large pretrained language models for adaptive clustering on contextualized relational features.
Outcome:	The proposed framework exploits weak, self-supervised signals on open-domain Relation Extraction . it bootstraps the self-supervised signals by improving contextualized features in relation classification .

Copied to clipboard

Song Dai, Yibo Yan, Jiamin Su, Zihao Dongfang, Yubo Gao, Yonghua Hei, Jungang Li, Junyan Zhang, Sicheng Tao, Zhuoran Gao, Xuming Hu

Automatic Table Union Search with Tabular Representation Learning (2023.findings-acl)

Challenge:	Current physics benchmarks focus on text-only inputs or only on problem-solving . current physics reasoning benchmarks neglect critical intermediate steps of variable identification and process formulation.
Approach:	a new benchmark evaluates multimodal large language models in physics reasoning . the benchmark measures variables, process formulations, and solution derivation .
Outcome:	PhysicsArena is the first multimodal physics reasoning benchmark . it evaluates MLLMs across three critical dimensions: variable identification, process formulation, and solution derivation.

Copied to clipboard

Xuming Hu, Shen Wang, Xiao Qin, Chuan Lei, Zhengyuan Shen, Christos Faloutsos, Asterios Katsifodimos, George Karypis, Lijie Wen, Philip S. Yu

Pierce the Mists, Greet the Sky: Decipher Knowledge Overshadowing via Knowledge Circuit Analysis (2025.emnlp-main)

Challenge:	Existing methods to identify uniability based on column representations are insufficient to reveal latent relational features to describe column relation between pair of columns.
Approach:	They propose a self-supervised table union search framework called AutoTUS to learn column relational representations in a multi-stage manner.
Outcome:	The proposed framework improves on the SOTA baseline and on real-world datasets.

Copied to clipboard

Haoming Huang, Yibo Yan, Jiahao Huo, Xin Zou, Xinfeng Li, Kun Wang, Xuming Hu

Data Whisperer: Efficient Data Selection for Task-Specific LLM Fine-Tuning via Few-Shot In-Context Learning (2025.acl-long)

Challenge:	Large Language Models (LLMs) are hampered by hallucinations, a particularly challenging variant, knowledge overshadowing, which can lead to erroneous outputs even with high-quality training data.
Approach:	They propose a framework to analyze and detect knowledge overshadowing by using knowledge circuit analysis to dissect the function of key components in the circuit and how attention pattern dynamics contribute to the phenomenon.
Outcome:	Extensive experiments show that the framework can detect and analyze knowledge overshadowing and improves on existing models.

Copied to clipboard

Shaobo Wang, Xiangqi Jin, Ziming Wang, Jize Wang, Jiajun Zhang, Kaixin Li, Zichen Wen, Zhong Li, Conghui He, Xuming Hu, Linfeng Zhang

CHEF: A Pilot Chinese Dataset for Evidence-Based Fact-Checking (2022.naacl-main)

Challenge:	Using fine-tuning on task-specific data is essential for large language models to be effective in specialized tasks.
Approach:	They propose a method that leverages few-shot in-context learning with the model to be fine-tuned.
Outcome:	The proposed method outperforms existing methods with a 3.1-point improvement and a 7.4 speedup on the Llama-3-8B-Instruct model using just 10% of the dataset.

Copied to clipboard

Xuming Hu, Zhijiang Guo, GuanYu Wu, Aiwei Liu, Lijie Wen, Philip Yu

HiURE: Hierarchical Exemplar Contrastive Learning for Unsupervised Relation Extraction (2022.naacl-main)

Challenge:	CHEF dataset provides evidence retrieval over non-English claims . e-fact-checking is a time-consuming task, which can take journalists several hours or days.
Approach:	They construct a dataset of 10K real-world claims that is based on annotated evidence retrieved from the Internet.
Outcome:	The proposed dataset provides evidence retrieval as a latent variable and can be used to train and reason over non-English claims.

Copied to clipboard

Shuliang Liu, Xuming Hu, Chenwei Zhang, Shu’ang Li, Lijie Wen, Philip Yu

GDA: Generative Data Augmentation Techniques for Relation Extraction Tasks (2023.findings-acl)

Challenge:	Existing methods to extract relational feature signals from natural language sentences use self-supervised clustering and classification that cause gradual drift problems.
Approach:	They propose a framework that derives hierarchical signals from relational feature space using cross hierarchy attention and effectively optimizes relation representation of sentences under exemplar-wise contrastive learning.
Outcome:	The proposed framework can extract the relationship between entities from natural language sentences without prior knowledge on relation scope or distribution.

Copied to clipboard

Xuming Hu, Aiwei Liu, Zeqi Tan, Xin Zhang, Chenwei Zhang, Irwin King, Philip S. Yu

Character-level White-Box Adversarial Attacks against Transformers via Attachable Subwords Substitution (2022.emnlp-main)

Challenge:	Existing work adopts data augmentation techniques to generate pseudo-annotated sentences . existing methods neither preserve semantic consistency of original sentences nor preserve syntax structure of sentences when expressing relations using seq2seq models, resulting in less diverse augmentations.
Approach:	They propose a dedicated augmentation technique for relational texts, named GDA, which uses two complementary modules to preserve both semantic consistency and syntax structures.
Outcome:	The proposed technique can bring 2.0% F1 improvements in three datasets under low-resource setting.

Copied to clipboard

Aiwei Liu, Honghai Yu, Xuming Hu, Shu’ang Li, Li Lin, Fukun Ma, Yawen Yang, Lijie Wen

Robustness in NLP

Challenge:	Existing methods to attack transformer models are not effective at character level, but they are a natural attack scenario.
Approach:	They propose a character-level adversarial attack method against transformer models . they use a gradient-based method to find the most vulnerable words in a sentence .
Outcome:	The proposed method outperforms previous methods on sentence-level and token-level tasks.

ErrorRadar: Benchmarking Complex Mathematical Reasoning of Multimodal Large Language Models Via Error Detection (2026.findings-acl)

Copied to clipboard

Yibo Yan, Shen Wang, Jiahao Huo, Hang Li, Boyan Li, Jiamin Su, Xiong Gao, YiFan Zhang, Tianlong Xu, Zhendong Chu, Aoxiao Zhong, Kun Wang, Hui Xiong, Philip S. Yu, Xuming Hu, Qingsong Wen

Domain-Specific NER via Retrieving Correlated Samples (2022.coling-1)

Challenge:	Current mathematical benchmarks focus on evaluating MLLMs’ problem-solving ability, yet there is a crucial gap in addressing more complex scenarios such as error detection.
Approach:	They propose to evaluate multimodal error detection by evaluating two sub-tasks error step identification and error categorization.
Outcome:	The proposed task evaluates MLLMs' ability to handle multimodal questions compared to text-only models.

Copied to clipboard

Xin Zhang, Yong Jiang, Xiaobin Wang, Xuming Hu, Yueheng Sun, Pengjun Xie, Meishan Zhang

Are We Using the Right Benchmark: An Evaluation Framework for Visual Token Compression Methods (2026.acl-long)

Challenge:	Successful Named Entity Recognition models fail on texts from some special domains, for example, Chinese addresses and e-commerce titles.
Approach:	They propose to enhance NER models with correlated samples to help the text understanding . they draw correlated texts by the sparse BM25 retriever from large-scale in-domain unlabeled data .
Outcome:	Empirical results show that NER models can be enhanced with correlated samples . the proposed model can be used to reason out the correct answer on hard cases .

Copied to clipboard

Chenfei Liao, Wensong Wang, Zichen Wen, Xu Zheng, Yiyu Wang, Haocong He, Yuanhuiyi Lyu, Lutao Jiang, Xin Zou, Yuqian Fu, Bin Ren, Linfeng Zhang, Xuming Hu

Unraveling Babel: Exploring Multilingual Activation Patterns of LLMs and Their Applications (2024.emnlp-main)

Challenge:	Recent efforts to accelerate inference in Multimodal Large Language Models have focused on visual token compression.
Approach:	They propose a framework that leverages downsampling as a discriminator to denoise existing benchmarks.
Outcome:	The proposed evaluation framework leverages downsampling as a discriminator to denoise existing benchmarks.

Copied to clipboard

Weize Liu, Yinlong Xu, Hongxia Xu, Jintai Chen, Xuming Hu, Jian Wu

Multilinguality

Challenge:	Recent studies have focused on how large language models process multiple languages, but internal mechanisms of LLMs remain insufficiently explored.
Approach:	They propose to convert dense LLMs into fine-grained MoE architectures and analyze their activation patterns using expert activation frequency heatmaps.
Outcome:	The proposed method outperforms random expert pruning and exceeds models in some languages.

Gradient Imitation Reinforcement Learning for Low Resource Relation Extraction (2021.emnlp-main)

Copied to clipboard

Xuming Hu, Chenwei Zhang, Yawen Yang, Xiaohe Li, Li Lin, Lijie Wen, Philip S. Yu

Low-Resource NLP

Challenge:	Existing methods to extract relation facts from limited labeled corpora are laborintensive to obtain . Existing approaches use self-training to generate pseudo labels that will cause gradual drift problem or leverage meta-learning scheme which does not solicit feedback explicitly.
Approach:	They propose a Gradient Imitation Reinforcement Learning method to encourage pseudo label data to imitate gradient descent direction on labeled data and bootstrap its optimization capability through trial and error.
Outcome:	The proposed method handles two major scenarios in low-resource relation extraction when no unlabeled data is available.

StreamMeCo: Long-Term Agent Memory Compression for Efficient Streaming Video Understanding (2026.findings-acl)

Copied to clipboard

Junxi Wang, Te Sun, Jiayi Zhu, Junxian Li, Haowen Xu, Zichen Wen, Xuming Hu, Zhiyu li, Linfeng Zhang

LLM Agents for Education: Advances and Applications (2025.findings-emnlp)

Challenge:	StreamMeCo is an efficient Stream Agent Memory Compression framework for video understanding.
Approach:	They propose an efficient Stream Agent Memory Compression framework that evicts redundant memory nodes and introduces a time-decay memory retrieval mechanism to mitigate performance degradation.
Outcome:	The proposed framework achieves 1.87 speedup in memory retrieval while delivering an average accuracy improvement of 1.0% on three challenging benchmark datasets.

Copied to clipboard

Zhendong Chu, Shen Wang, Jian Xie, Tinghui Zhu, Yibo Yan, Jingheng Ye, Aoxiao Zhong, Xuming Hu, Jing Liang, Philip S. Yu, Qingsong Wen

Entity-to-Text based Data Augmentation for various Named Entity Recognition Tasks (2023.findings-acl)

Challenge:	Large Language Model (LLM) agents are transforming education by automating complex tasks and enhancing both teaching and learning processes.
Approach:	This survey analyzes recent advances in applying Large Language Model agents to educational settings . it highlights ethical issues, hallucination and overreliance, and integration with existing ecosystems .
Outcome:	The authors analyze the technologies enabling LLM agents and highlight key challenges in deploying them in educational settings.

Copied to clipboard

Xuming Hu, Yong Jiang, Aiwei Liu, Zhongqiang Huang, Pengjun Xie, Fei Huang, Lijie Wen, Philip S. Yu

EssayJudge: A Multi-Granular Benchmark for Assessing Automated Essay Scoring Capabilities of Multimodal Large Language Models (2025.findings-acl)

Challenge:	Existing augmentation techniques manipulate words in the original text that break the semantic coherence of the text, or exploit generative models that ignore preserving entities in the text.
Approach:	They propose a novel Entity-to-Text based data augmentation technique called EnTDA to add, delete, replace or swap entities in the original text.
Outcome:	The proposed technique generates semantically coherent and entity preserving texts on thirteen NER tasks and two settings.

Copied to clipboard

Jiamin Su, Yibo Yan, Fangteng Fu, Zhang Han, Jingheng Ye, Xiang Liu, Jiahao Huo, Huiyu Zhou, Xuming Hu

Exploring Response Uncertainty in MLLMs: An Empirical Evaluation under Misleading Scenarios (2025.emnlp-main)

Challenge:	Automated Essay Scoring (AES) systems face three major challenges: reliance on handcrafted features that limit generalizability, difficulty in capturing fine-grained traits like coherence and argumentation, and inability to handle multimodal contexts.
Approach:	They propose a multimodal benchmark to evaluate AES capabilities across lexical-, sentence-, and discourse-level traits without manual feature engineering.
Outcome:	The proposed system can evaluate AES capabilities across lexical-, sentence-, and discourse-level traits without manual feature engineering.

Copied to clipboard

Yunkai Dang, Mengxi Gao, Yibo Yan, Xin Zou, Yanggan Gu, Jungang Li, Jingyu Wang, Peijie Jiang, Aiwei Liu, Jia Liu, Xuming Hu

Rethinking the Roles of Large Language Models in Chinese Grammatical Error Correction (2025.acl-industry)

Challenge:	Existing studies have focused mainly on visual–textual misalignment, leaving largely unexplored the MLLMs’ ability to preserve an original correct answer when confronted with misleading information.
Approach:	They propose a two-stage evaluation pipeline to quantify the response uncertainty phenomenon by eliciting each model’s original response on unperturbed inputs and injecting explicit (false-answer hints) and implicit (contextual contradictions) misleading instructions.
Outcome:	The proposed model overturns a correct answer in 65% of cases after receiving a single deceptive cue.

Copied to clipboard

Yinghui Li, Shang Qin, Jingheng Ye, Haojing Huang, Yangning Li, Shu-Yu Guo, Libo Qin, Xuming Hu, Wenhao Jiang, Hai-Tao Zheng, Philip S. Yu

Thinking Economically: A Hierarchical Framework for Adaptive-Complexity Reasoning in LLMs (2026.findings-acl)

Challenge:	Recent studies have shown that Large Language Models’ performance as correctors on Chinese Grammatical Error Correction (CGEC) remains unsatisfactory due to the challenging nature of the task.
Approach:	They propose a training framework EXAM that uses LLMs as explainers to enhance CGEC small models and a novel evaluation method SEE that utilizes LLM as evaluators to bring more reasonable evaluations.
Outcome:	The proposed methods improve the performance of LLMs on Chinese Grammatical Error Correction (CGEC) task.

Copied to clipboard

Yubo Gao, Haotian Wu, Hong Chen, Junquan Huang, Yibo Yan, Jungang Li, Zihao Dongfang, Sicheng Tao, PS Tan, Jie Zhang, Xuming Hu

Green & Sustainable NLP

Challenge:	Existing efficiency methods for Chain-of-Thought (CoT) generate excessively long rationales without commensurate accuracy gains.
Approach:	They propose a training framework that operationalizes this principle through coarse-to-fine budgeting.
Outcome:	Experiments on GSM8K and MATH500 show that HAB surpasses standard CoT in accuracy and reduces token usage, achieving stronger performance-efficiency trade-off than baselines.

Dynamic Expert Specialization: Towards Catastrophic Forgetting-Free Multi-Domain MoE Adaptation (2025.emnlp-main)

Copied to clipboard

Junzhuo Li, Bo Wang, Xiuze Zhou, Xuming Hu

PolyJoin: Semantic Multi-key Joinable Table Search in Data Lakes (2025.findings-naacl)

Challenge:	Existing approaches to adapt Mixture-of-Experts models to multiple domains are prohibitive computation, cross-domain interference or require separate runs per domain.
Approach:	They propose a dynamic expert specialization framework for multi-domain adaptation of Mixture-of-Experts models.
Outcome:	The proposed framework reduces forgetting by 89% compared to full fine-tuning as domains scale from 2 to 6 and achieves faster convergence than conventional methods.

Copied to clipboard

Xuming Hu, Chuan Lei, Xiao Qin, Asterios Katsifodimos, Christos Faloutsos, Huzefa Rangwala

Capturing Nuanced Preferences: Preference-Aligned Distillation for Small Language Models (2025.findings-acl)

Challenge:	Existing joinable table search methods focus on single key (unary) joins, where a single column is the join key, but are ineffective when dealing with join keys composed of multiple columns (n-ary joins) Existing methods are inefficient when dealing . with joins composed of n-aries, which are prevalent on web table corpora.
Approach:	They propose a joinable table search method that finds multi-key joinable tables on the web, given a query table.
Outcome:	The proposed method outperforms the state-of-the-art methods on two real-world web table benchmarks.

Copied to clipboard

Yanggan Gu, Junzhuo Li, Sirui Huang, Xin Zou, Zhenghua Li, Xuming Hu

On the Robustness of Document-Level Relation Extraction Models to Entity Name Variations (2024.findings-acl)

Challenge:	Existing methods for aligning small language models with human values model preference knowledge from large language models (LLMs) however, this limitation hinders student SLMs from capturing nuanced preferences for multiple responses.
Approach:	They propose a framework which models teacher's preference knowledge as a probability distribution over all potential preferences, thereby providing more nuanced supervisory signals.
Outcome:	The proposed framework outperforms existing methods on four benchmark tasks and achieves 20% improvement on AlpacaEval 2 and Arena-Hard.

Copied to clipboard

Shiao Meng, Xuming Hu, Aiwei Liu, Fukun Ma, Yawen Yang, Shuang Li, Lijie Wen

StructFact: Reasoning Factual Knowledge from Structured Data with Large Language Models (2025.findings-acl)

Challenge:	Existing DocRE models which perform well may make more mistakes when merely changing the entity names in the document, hindering the generalization to novel entity names.
Approach:	They propose a pipeline to generate entity-renamed documents by replacing the original entity names with names from Wikidata.
Outcome:	The proposed pipeline generates entity-renamed documents by replacing the original entity names with names from Wikidata.

Copied to clipboard

Sirui Huang, Yanggan Gu, Zhonghao Li, Xuming Hu, Li Qing, Guandong Xu

Can LLM Watermarks Robustly Prevent Unauthorized Knowledge Distillation? (2025.acl-long)

Challenge:	Large language models (LLMs) have made significant strides in natural language processing by leveraging their ability to comprehend and reason with factual knowledge.
Approach:	They propose a benchmark to evaluate the ability of large language models to reason with structured data for knowledge-intensive tasks.
Outcome:	Extensive tests on 10 common LLMs show that they struggle with heterogeneity of structured data during reasoning.

Copied to clipboard

Leyi Pan, Aiwei Liu, Shiyu Huang, Yijian Lu, Xuming Hu, Lijie Wen, Irwin King, Philip S. Yu

DeKeyNLU: Enhancing Natural Language to SQL Generation through Task Decomposition and Keyword Extraction (2025.findings-emnlp)

Challenge:	Large Language Model (LLM) watermarking is radioactive and enables the detection of watermarks inherited by student models when trained on the outputs of watermarked teacher models.
Approach:	They propose two types of watermark removal attacks that allow student models to perform untraceable knowledge distillation while avoiding watermark inheritance.
Outcome:	The proposed attacks eliminate inherited watermarks while maintaining knowledge transfer efficiency and low computational overhead.

Copied to clipboard

Jian Chen, Zhenyan Chen, Xuming Hu, Peilin Zhou, Yining Hua, Han Fang, Cissy Hing Yee Choy, Xinmei Ke, Jingfeng Luo, Zixuan Yuan

Programming Languages in NLP

Challenge:	NL2SQL provides a model-centric paradigm that simplifies database access for non-technical users . challenges such as inaccurate task decomposition and keyword extraction remain major bottlenecks .
Approach:	They propose a RAG-based NL2SQL pipeline that employs three modules for query understanding, entity retrieval, and generation to improve SQL generation accuracy.
Outcome:	The proposed pipeline improves the accuracy of query generation on BIRD and Spider datasets.

MMNeuron: Discovering Neuron-Level Domain-Specific Interpretation in Multimodal Large Language Model (2024.emnlp-main)

Copied to clipboard

Jiahao Huo, Yibo Yan, Boren Hu, Yutao Yue, Xuming Hu

Optimal Expert-Attention Allocation in Mixture-of-Experts: A Scalable Law for Dynamic Model Design (2026.acl-industry)

Challenge:	Existing MLLMs have a visual question answering capability but lack domain-specific information.
Approach:	They propose a framework for language model modules in MLLMs when handling projected image features and verify this hypothesis using logit lens.
Outcome:	The proposed framework will yield a 10% change in accuracy at most, shedding light on the development of cross-domain, all-encompassing MLLMs in the future.

Copied to clipboard

Junzhuo Li, Peijie Jiang, Changxin Tian, Jia Liu, Zhiqiang Zhang, Xuming Hu

Green & Sustainable NLP

Challenge:	a novel extension of neural scaling laws to Mixture-of-Experts models is proposed . a ratio of expert-attention compute is crucial for efficient MoE models .
Approach:	They propose an extension of neural scaling laws to Mixture-of-Experts (MoE) models . they define the ratio r as the fraction of total FLOPs per token dedicated to expert and attention layers .
Outcome:	The proposed model can be tuned beyond size and data with the proposed model.

RAPL: A Relation-Aware Prototype Learning Approach for Few-Shot Document-Level Relation Extraction (2023.emnlp-main)

Copied to clipboard

Shiao Meng, Xuming Hu, Aiwei Liu, Shuang Li, Fukun Ma, Yawen Yang, Lijie Wen

Query-based Instance Discrimination Network for Relational Triple Extraction (2022.emnlp-main)

Challenge:	Existing methods to identify semantic relations between entities are time-consuming and labor-intensive.
Approach:	They propose a relation-aware prototype learning method for document-level relation extraction (FSDLRE) they propose RAPL, which judiciously leverages relation descriptions and real NOTA instances as guidance .
Outcome:	The proposed method outperforms state-of-the-art approaches by 2.61% F1 . it generates task-specific NOTA prototypes and refines relation prototypes .

Copied to clipboard

Zeqi Tan, Yongliang Shen, Xuming Hu, Wenqi Zhang, Xiaoxia Cheng, Weiming Lu, Yueting Zhuang

Vision-Language Introspection: Mitigating Overconfident Hallucinations in MLLMs via Interpretable Bi-Causal Steering (2026.acl-long)

Challenge:	Recent approaches to extract relational triples from open domain texts suffer from error propagation, relation redundancy and lack of high-level connections.
Approach:	They propose a query-based approach to construct instance-level representations for relational triples . they use query embeddings and token embeddables to extract all types of triples in one step .
Outcome:	The proposed method achieves state-of-the-art on five widely used benchmarks.

Copied to clipboard

Shuliang Liu, Songbo Yang, Dong Fang, Sihang Jia, Yuqi Tang, Lingfeng Su, Ruoshui Peng, Yibo Yan, Xin Zou, Xuming Hu

Decoding Knowledge Attribution in Mixture-of-Experts: A Framework of Basic-Refinement Collaboration and Efficiency Analysis (2025.acl-long)

Challenge:	Existing approaches to overcome object hallucination are limited . Existing mitigations include costly retraining and a training-free inference framework .
Approach:	They propose a training-free inference framework that simulates a metacognitive self-correction process.
Outcome:	The proposed framework reduces object hallucination rates by 12.67% on MMHal-Bench and improves accuracy by 5.8% on POPE.

Copied to clipboard

Junzhuo Li, Bo Wang, Xiuze Zhou, Peijie Jiang, Jia Liu, Xuming Hu

MarkLLM: An Open-Source Toolkit for LLM Watermarking (2024.emnlp-demo)

Challenge:	Existing attribution methods for dense models fail to capture dynamic routing-expert interactions in sparse MoE architectures.
Approach:	They propose to analyze sparse MoE architectures against dense models to capture dynamic routing-expert interactions.
Outcome:	The proposed algorithm shows that sparse models achieve higher efficiency per layer . it also shows that deep Qwen-MoE mitigates expert failures while minimizing complexity .

Copied to clipboard

Leyi Pan, Aiwei Liu, Zhiwei He, Zitian Gao, Xuandong Zhao, Yijian Lu, Binglin Zhou, Shuliang Liu, Xuming Hu, Lijie Wen, Irwin King, Philip Yu

Do BERT-Like Bidirectional Models Still Perform Better on Text Classification in the Era of LLMs? (2025.findings-emnlp)

Challenge:	Large Language Models (LLMs) embed imperceptible yet algorithmically detectable signals in outputs to identify LLM-generated text.
Approach:	They propose to develop an open-source toolkit for LLM watermarking that embeds imperceptible yet algorithmically detectable signals in model outputs to identify LLM-generated text.
Outcome:	MarkLLM provides a unified framework for implementing LLM watermarking algorithms, while providing user-friendly interfaces to ensure ease of access.

Copied to clipboard

Junyan Zhang, Yiming Huang, Shuliang Liu, Yubo Gao, Xuming Hu

LLMArena: Assessing Capabilities of Large Language Models in Dynamic Multi-Agent Environments (2024.acl-long)

Challenge:	Rapid adoption of LLMs has overshadowed the potential advantages of traditional BERT-like models in text classification.
Approach:	They compare BERT-like models fine-tuning, LLM internal state utilization, and LLM zero-shot inference across six datasets.
Outcome:	The proposed method outperforms LLMs on six challenging datasets.

Copied to clipboard

Junzhe Chen, Xuming Hu, Shuodi Liu, Shiyu Huang, Wei-Wei Tu, Zhaofeng He, Lijie Wen

Sculpting the Vector Space: Towards Efficient Multi-Vector Visual Document Retrieval via Prune-then-Merge Framework (2026.findings-acl)

Challenge:	Existing benchmarks for evaluating large language models use static datasets, leading to data leakage or overlooking the complexities of multi-agent interactions.
Approach:	They propose a framework that evaluates the diverse capabilities of LLM agents in multi-agent dynamic environments.
Outcome:	The proposed framework assesses the diverse capabilities of LLM agents in multi-agent dynamic environments.

Copied to clipboard

Yibo Yan, Mingdong Ou, Yi Cao, Xin Zou, Jiahao Huo, Shuliang Liu, James Kwok, Xuming Hu

Information Retrieval

Challenge:	Visual Document Retrieval (VDR) is of importance in multimodal retrieval applications.
Approach:	They propose a two-stage pruning and merging frameworks that combine pruning and merge techniques to achieve higher compression rates.
Outcome:	The proposed framework outperforms existing methods on 29 visual document retrieval datasets.

Internal Chain-of-Thought: Empirical Evidence for Layer‐wise Subtask Scheduling in LLMs (2025.emnlp-main)

Copied to clipboard

Zhipeng Yang, Junzhuo Li, Siyu Xia, Xuming Hu

Challenge:	et al., 2024) show large language models exhibit internal chain-of-thought, meaning they decompose and execute composite tasks layer-by-layer.
Approach:	They propose a method to decode hidden states using LogitLens . they also propose 'chain of thought' to decompose and execute composite tasks .
Outcome:	The proposed method decodes hidden states and shows consistent execution pattern . it opens avenues for fine-grained, instruction-level activation steering.

A Survey on Proactive Defense Strategies Against Misinformation in Large Language Models (2025.findings-acl)

Copied to clipboard

Shuliang Liu, Hongyi Liu, Aiwei Liu, Duan Bingchen, Zheng Qi, Yibo Yan, He Geng, Peijie Jiang, Jia Liu, Xuming Hu

AMR-based Network for Aspect-based Sentiment Analysis (2023.acl-long)

Challenge:	Existing methods for detection of misinformation generated by large language models fail to mitigate societal risks . authors propose a paradigm shift from passive detection to anticipatory mitigation strategies . existing defenses remain reactionary in an era demanding proactive defense, authors say .
Approach:	They propose a three-pillar approach to prevent misinformation by fortifying integrity of training data and inference reliability by embedding self-corrective mechanisms during reasoning.
Outcome:	The proposed framework improves existing methods in misinformation prevention by 63% . it demonstrates that existing methods exhibit false negative rates against misinformation .

Copied to clipboard

Fukun Ma, Xuming Hu, Aiwei Liu, Yawen Yang, Shuang Li, Philip S. Yu, Lijie Wen

Refiner: Restructure Retrieved Content Efficiently to Advance Question-Answering Capabilities (2024.findings-emnlp)

Challenge:	Recent studies have used dependency trees to extract relation between aspects and contexts, but there is a potential mismatch between the dependency tree and sentiment classification as a semantic task.
Approach:	They propose to replace the syntactic dependency tree with a semantic structure to capture the relation between an aspect and a context.
Outcome:	The proposed model improves ABSA on four public datasets with 1.13% improvement over baselines.

Copied to clipboard

Zhonghao Li, Xuming Hu, Aiwei Liu, Kening Zheng, Sirui Huang, Hui Xiong

Green & Sustainable NLP

Challenge:	Large Language Models (LLMs) are limited by their parametric knowledge, leading to hallucinations in knowledge-extensive tasks.
Approach:	They propose an end-to-end extract-and-restructure paradigm that leverages a single decoder-only LLM to adaptively extract query-relevant contents verbatim along with the necessary context.
Outcome:	Experiments show that a trained Refiner outperforms state-of-the-art RAG and compressing approaches in multiple tasks.

Mind’s Mirror: Distilling Self-Evaluation Capability and Comprehensive Thinking from Large Language Models (2024.naacl-long)

Copied to clipboard

Weize Liu, Guocong Li, Kai Zhang, Bang Du, Qiyuan Chen, Xuming Hu, Hongxia Xu, Jintai Chen, Jian Wu

A Survey of Mathematical Reasoning in the Era of Multimodal Large Language Model: Benchmark, Method & Challenges (2025.findings-acl)

Challenge:	Large language models (LLMs) have achieved significant advances in natural language processing, but their scale and computational demands pose challenges to their practical application.
Approach:	They propose a method for distilling the self-evaluation capability from LLMs into SLMs and advocate for more comprehensive thinking by incorporating multiple distinct CoTs and self-estimation outputs.
Outcome:	The proposed method significantly improves the performance of distilled SLMs on three NLP benchmarks.

Copied to clipboard

Yibo Yan, Jiamin Su, Jianxiang He, Fangteng Fu, Xu Zheng, Yuanhuiyi Lyu, Kun Wang, Shen Wang, Qingsong Wen, Xuming Hu

MathAgent: Leveraging a Mixture-of-Math-Agent Framework for Real-World Multimodal Mathematical Error Detection (2025.acl-industry)

Challenge:	This survey provides the first comprehensive analysis of mathematical reasoning in the era of multimodal large language models . integrating large language model with mathematical reasoning tasks is becoming significant as AI advances .
Approach:	They review over 200 studies published since 2021 and examine the state-of-the-art developments in Math-LLMs . they identify five major challenges hindering the realization of AGI in this domain .
Outcome:	The authors examine the state-of-the-art developments in Math-LLMs with a focus on multimodal settings.

Copied to clipboard

Yibo Yan, Shen Wang, Jiahao Huo, Philip S. Yu, Xuming Hu, Qingsong Wen

SafeEraser: Enhancing Safety in Multimodal Large Language Models through Multimodal Machine Unlearning (2025.findings-acl)

Challenge:	Multimodal Large Language Models (MLLMs) struggle with identifying and categorizing student errors in multimodal mathematical contexts.
Approach:	They propose a new framework that decomposes error detection into three phases with specialized agents.
Outcome:	The proposed framework shows higher accuracy in error step identification and 3% improvement in error categorization on real-world educational data.

Copied to clipboard

Junkai Chen, Zhijie Deng, Kening Zheng, Yibo Yan, Shuliang Liu, PeiJun Wu, Peijie Jiang, Jia Liu, Xuming Hu

MMUnlearner: Reformulating Multimodal Machine Unlearning in the Era of Multimodal Large Language Models (2025.findings-acl)

Challenge:	Existing methods for MU forget quality and model utility are not fully explored for safety in MLLMs.
Approach:	They propose a safety unlearning benchmark for MLLMs to measure over-forgetting . they propose MU methods to forget quality and model utility .
Outcome:	The proposed method reduces over-forgetting by 79.5% while maintaining forget quality and model utility.

Copied to clipboard

Jiahao Huo, Yibo Yan, Xu Zheng, Yuanhuiyi Lyu, Xin Zou, Zhihua Wei, Xuming Hu

Evaluating Robustness of Generative Search Engine on Adversarial Factoid Questions (2024.findings-acl)

Challenge:	Recent advances in machine learning (MU) have enabled the selective removal of private or sensitive information encoded within deep neural networks.
Approach:	They propose to "reformulate" the task of multimodal MU in the era of MLLMs by preserving only the visual patterns associated with a given entity while preserving the corresponding textual knowledge.
Outcome:	The proposed method surpasses baselines that finetuned MLLMs with VQA data directly through Gradient Ascent (GA) or Negative Preference Optimization (NPO), across all evaluation dimensions.

Copied to clipboard

Xuming Hu, Xiaochuan Li, Junzhe Chen, Yinghui Li, Yangning Li, Xiaoguang Li, Yasheng Wang, Qun Liu, Lijie Wen, Philip Yu, Zhijiang Guo

Robustness in NLP

Challenge:	Existing large language models (LLMs)-backed generative search engines may not always be accurate.
Approach:	They propose to evaluate the robustness of retrieval-augmented generation in a realistic and high-risk setting where adversaries have only black-box system access.
Outcome:	The proposed model exhibits higher susceptibility to factual errors compared to LLMs without retrieval.

Reefknot: A Comprehensive Benchmark for Relation Hallucination Evaluation, Analysis and Mitigation in Multimodal Large Language Models (2025.findings-acl)

Copied to clipboard

Kening Zheng, Junkai Chen, Yibo Yan, Xin Zou, Huiyu Zhou, Xuming Hu

Enhancing Cross-lingual Natural Language Inference by Soft Prompting with Multilingual Verbalizer (2023.findings-acl)

Challenge:	Existing research focuses on object-level or attribute-level hallucinations, neglecting the more complex relation hallucinosities.
Approach:	They propose a comprehensive benchmark targeting relation hallucinations comprising over 20,000 real-world samples and a confidence-based mitigation strategy which reduces the halluciation rate by an average of 9.75% across three datasets.
Outcome:	The proposed approach reduces the hallucination rate by an average of 9.75% across three datasets, including Reefknot.

Copied to clipboard

Shuang Li, Xuming Hu, Aiwei Liu, Yawen Yang, Fukun Ma, Philip S. Yu, Lijie Wen

Low-Resource NLP

Challenge:	Existing approaches to cross-lingual natural language inference lack annotated parallel corpora.
Approach:	They propose a new prompt learning framework with the Multilingual Verbalizer for XNLI that uses a multilingual verbalizer to align the representations of original and augmented multilingual questions into a unified semantic space with consistency regularization.
Outcome:	The proposed framework outperforms existing methods under few-shot and full-shot cross-lingual transfer settings.

Unlocking Speech Instruction Data Potential with Query Rewriting (2025.findings-acl)

Copied to clipboard

Yonghua Hei, Yibo Yan, Shuliang Liu, Huiyu Zhou, Linfeng Zhang, Xuming Hu

Text Generation

Challenge:	Existing LLMs lack datasets and biased training tasks to follow speech instructions.
Approach:	They propose a query rewriting framework that uses multiple agents to annotate and validate the synthesized speech.
Outcome:	The proposed framework can transform text instructions into distributions more suitable for TTS models for speech synthesis without human annotation.

Scene Graph Modification as Incremental Structure Expanding (2022.coling-1)

Copied to clipboard

Xuming Hu, Zhijiang Guo, Yu Fu, Lijie Wen, Philip S. Yu

UAlign: Leveraging Uncertainty Estimations for Factuality Alignment on Large Language Models (2025.acl-long)

Challenge:	Scene graphs are used in cross-modal tasks such as image retrieval, image captioning, and visual question answering.
Approach:	They propose a model that iterates between nodes prediction and edges prediction . they frame scene graph modification as a graph expansion task by introducing incremental structure expanding .
Outcome:	The proposed model surpasses the state-of-the-art model by large margins on four benchmarks.

Copied to clipboard

Boyang Xue, Fei Mi, Qi Zhu, Hongru Wang, Rui Wang, Sheng Wang, Erxin Yu, Xuming Hu, Kam-Fai Wong

Semi-supervised Relation Extraction via Incremental Meta Self-Training (2021.findings-emnlp)

Challenge:	Large Language Models (LLMs) often struggle to accurately express factual knowledge, especially in cases where the knowledge boundaries are ambiguous.
Approach:	They propose a framework that leverages Uncertainty estimations to represent knowledge boundaries and incorporates these representations into prompts for LLMs to Align with factual knowledge.
Outcome:	The proposed framework significantly improves the LLMs’ capacities to confidently answer known questions and refuse unknown questions on both in-domain and out-of-domain tasks.

Copied to clipboard

Xuming Hu, Chenwei Zhang, Fukun Ma, Chenyao Liu, Lijie Wen, Philip S. Yu

LongGenBench: Long-context Generation Benchmark (2024.findings-emnlp)

Challenge:	Existing methods suffer from the gradual drift problem, where noisy pseudo labels are incorporated during training.
Approach:	They propose a method that uses pseudo labels to assess quality on unlabeled samples . they use a relation label generation network to learn from successful and failed attempts .
Outcome:	Experimental results show the proposed method can improve on two public datasets.

Copied to clipboard

Xiang Liu, Peijie Dong, Xuming Hu, Xiaowen Chu

Multimodal Relation Extraction with Cross-Modal Retrieval and Synthesis (2023.acl-short)

Challenge:	Current long-context benchmarks focus on retrieval-based tests, requiring Large Language Models to locate specific information within extensive input contexts.
Approach:	They propose a long-context generation benchmark that allows for flexible configurations of customized generation context lengths.
Outcome:	The proposed benchmark improves performance on NIAH and other retrieval-based tests.

Copied to clipboard

Xuming Hu, Zhijiang Guo, Zhiyang Teng, Irwin King, Philip S. Yu

DiscoverGPT: Multi-task Fine-tuning Large Language Model for Related Table Discovery (2025.findings-naacl)

Challenge:	Existing retrieval-augmented approaches focus on modeling the retrieved textual knowledge but this may not be able to accurately identify complex relations.
Approach:	They propose to retrieve multimodal relation extraction information based on object, sentence, and whole image . they propose to synthesize the object-level, image-level and sentence-level information .
Outcome:	The proposed method outperforms state-of-the-art models on multimodal relation extraction.

Copied to clipboard

Xuming Hu, Xiao Qin, Chuan Lei, Asterios Katsifodimos, Zhengyuan Shen, Balasubramaniam Srinivasan, Huzefa Rangwala