Papers by Xiao Xiong

LoRAMoE: Alleviating World Knowledge Forgetting in Large Language Models via MoE-Style Plugin (2024.acl-long)

Challenge:	Large language models exhibit significant performance discrepancies between high- and low-resource languages.
Approach:	They present an open-source multilingual LLM with 8 billion parameters and a multilingual instruction dataset.
Outcome:	The proposed model achieves consistent multilingual representations across languages.

Copied to clipboard

Shihan Dou, Enyu Zhou, Yan Liu, Songyang Gao, Wei Shen, Limao Xiong, Yuhao Zhou, Xiao Wang, Zhiheng Xi, Xiaoran Fan, Shiliang Pu, Jiang Zhu, Rui Zheng, Tao Gui, Qi Zhang, Xuanjing Huang

MINER: Improving Out-of-Vocabulary Named Entity Recognition from an Information Theoretic Perspective (2022.acl-long)

Challenge:	Experimental results show that, as the instruction data increases, LoRAMoE can significantly improve the ability to process downstream tasks, while maintaining the world knowledge stored in the LLM.
Approach:	They propose a framework that introduces several low-rank adapters and integrates them by using a router network to freeze the backbone model and force a portion of LoRAs to focus on leveraging world knowledge to solve downstream tasks.
Outcome:	The proposed framework freezes the backbone model and forces a portion of LoRAs to focus on leveraging world knowledge to solve downstream tasks, to alleviate world knowledge forgetting.

Copied to clipboard

Xiao Wang, Shihan Dou, Limao Xiong, Yicheng Zou, Qi Zhang, Tao Gui, Liang Qiao, Zhanzhan Cheng, Xuanjing Huang

Information Extraction & Text Mining

Challenge:	Named Entity Recognition models are feature-engineering and machine learning based.
Approach:	They propose a new NER learning framework that uses entity mentions to improve model performance.
Outcome:	The proposed model achieves better performance on OOV entities on various settings and datasets.

Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Models (2026.findings-acl)

Copied to clipboard

Hengyuan Zhang, Zhihao Zhang, Ercong Nie, Mingyang Wang, Zunhai Su, Yiwei Wang, Qianli Wang, Shuzhou Yuan, Xufeng Duan, Qibo Xue, Zeping Yu, Chenming Shang, Xiao Liang, Jing Xiong, Hui Shen, Chaofan Tao, Zhengwu Liu, Senjie Jin, Zhiheng Xi, Dongdong Zhang, Sophia Ananiadou, Tao Gui, Ruobing Xie, Hayden Kwok-Hay So, Hinrich Schuetze, Xuanjing Huang, Qi Zhang, Ngai Wong

Explainability & Interpretability in NLP

Challenge:	Existing literature on mechanistic interpretation (MI) treats it as an observational science, leaving practical applications underexplored.
Approach:	They propose a survey structured around the pipeline to identify and improve MI models.
Outcome:	The proposed framework enables tangible improvements in Alignment, Capability, and Efficiency.

Is EEG-to-Text Feasible in Real-World Scenarios? An In-Depth Analysis Using a Neuropsychology-Inspired Benchmark (2026.acl-long)

Copied to clipboard

Zihan Zhang, Yu Bao, Xiao Ding, Tianyi Jiang, Kai Xiong

Challenge:	Existing benchmarks for EEG2Text have neglected EEG instability, a problem that has confounded inference and sparked debate.
Approach:	They propose to use a 128-channel high-density EEG cap to evaluate EEG2Text models . they find existing benchmarks have neglected EEG instability, a flaw that has confounded inferences and sparked debate .
Outcome:	The proposed benchmarks provide key evidence for teacher-forcing-free decoding of EEG2Text models.

Consolidation or Adaptation? PRISM: Disentangling SFT and RL Data via Gradient Concentration (2026.acl-long)

Copied to clipboard

Yang Zhao, Yangou Ouyang, Xiao Ding, Hepeng Wang, Bibo Cai, Kai Xiong, Jinglong Gao, Zhouhao Sun, Li Du, Bing Qin, Ting Liu

A Confidence-based Partial Label Learning Model for Crowd-Annotated Named Entity Recognition (2023.findings-acl)

Challenge:	Existing data arbitration strategies for large language model training rely on surface-level heuristics that fail to diagnose intrinsic learning needs.
Approach:	They propose a framework that arbitrates data based on its degree of cognitive conflict with the model's existing knowledge.
Outcome:	Extensive experiments on WebShop and ALFWorld show that PRISM outperforms state-of-the-art hybrid methods while reducing computational costs by up to 3.22 .

Copied to clipboard

Limao Xiong, Jie Zhou, Qunxi Zhu, Xiao Wang, Yuanbin Wu, Qi Zhang, Tao Gui, Xuanjing Huang, Jin Ma, Ying Shan

Information Extraction & Text Mining

Challenge:	Existing models for named entity recognition (NER) are based on large-scale labeled datasets, which always obtain using crowdsourcing.
Approach:	They propose a CONfidence-based partial Label Learning method to integrate prior and posterior confidences for crowd-annotated named entity recognition models.
Outcome:	The proposed model improves on real-world and synthetic datasets compared with baselines.

Copied to clipboard

Yang Zhao, Li Du, Xiao Ding, Yangou Ouyang, Hepeng Wang, Kai Xiong, Jinglong Gao, Zhouhao Sun, Dongliang Xu, Qing Yang, Dongchen Li, Bing Qin, Ting Liu

Examining Inter-Consistency of Large Language Models Collaboration: An In-depth Analysis via Debate (2023.findings-emnlp)

Challenge:	Existing methods for selecting training data from general datasets fail to account for the joint distribution of instructions, resulting in inefficient learning and suboptimal knowledge transfer.
Approach:	They propose a method that constructs a mixed gradient-based instruction graph to capture the joint distribution and interdependencies among instructions.
Outcome:	The proposed method outperforms existing methods on domain adaptation tasks and in complex, data-scarce scenarios.

Copied to clipboard

Kai Xiong, Xiao Ding, Yixin Cao, Ting Liu, Bing Qin

Shiyuan Xiao, Yuanxin Ouyang, Wenge Rong, Jianxin Yang, Zhang Xiong

Challenge:	Existing studies focus on inconsistency issues within a single LLM, while we explore the inter-consistencies among multiple LLMs for collaboration.
Approach:	They propose a formal debate framework to examine whether LLMs can collaborate effectively to achieve a consensus for a shared goal.
Outcome:	The proposed framework enables LLMs to achieve consensus in three real-world debate scenarios with real-time scenarios aligned to the LLM's goals.

Copied to clipboard

Information Extraction & Text Mining

Challenge:	Named entity recognition (NER) tasks are a fundamental challenge for name recognition tasks that aim to reduce the boundary error when entities become longer.
Approach:	They propose a similarity based auxiliary classifier which can distinguish entity words from non-entity words by using vectors to indicate tags.
Outcome:	Empirical results show that the proposed classifier can perform better than baseline approaches.

Anti-Length Shift: Dynamic Outlier Truncation for Training Efficient Reasoning Models (2026.acl-long)

Copied to clipboard

Wei Wu, Liyi Chen, Congxi Xiao, Tianfu Wang, Qimeng Wang, Chengqiang Lu, Yan Gao, null Yiwu, Yao Hu, Hui Xiong

MegaPairs: Massive Data Synthesis for Universal Multimodal Retrieval (2025.acl-long)

Challenge:	Existing efficient reasoning methods rely on explicit length penalties for excessive verbosity on simple queries.
Approach:	They propose a training-time intervention that selectively suppresses redundant tokens . they find length shift occurs when models generate unnecessary reasoning on trivial inputs - a phenomenon that is often unexplored .
Outcome:	The proposed method reduces inference token usage by 78% while increasing accuracy compared to the initial policy and surpasses state-of-the-art efficient reasoning methods.

Copied to clipboard

Junjie Zhou, Yongping Xiong, Zheng Liu, Ze Liu, Shitao Xiao, Yueze Wang, Bo Zhao, Chen Jason Zhang, Defu Lian

Multimodality

Challenge:	despite the growing demand for multimodal retrieval, there is a lack of training data.
Approach:	They propose a data synthesis method that leverages vision language models and open-domain images to generate high-quality data.
Outcome:	The proposed method outperforms baseline models on 70 more datasets and can scale up.

ChatSOP: An SOP-Guided MCTS Planning Framework for Controllable LLM Dialogue Agents (2025.acl-long)

Copied to clipboard

Zhigen Li, Jianxiang Peng, Yanmeng Wang, Yong Cao, Tianhao Shen, Minghui Zhang, Linxi Su, Shang Wu, Yihang Wu, YuQian Wang, Ye Wang, Wei Hu, Jianfeng Li, Shaojun Wang, Jing Xiao, Deyi Xiong

Dialogue Systems & Conversational Agents

Challenge:	Existing models that use Large Language Models (LLMs) show superior performance in various tasks, but lack of controllability leads to unfocused conversations or task failure.
Approach:	They propose a standard operating procedure (SOP) framework to regulate dialogue flow by integrating Chain of Thought reasoning and supervised fine-tuning for SOP prediction.
Outcome:	The proposed method achieves a 27.95% improvement in action accuracy compared to baseline models based on GPT-3.5 and also shows notable gains for open-source models.

VISTA: Visualized Text Embedding For Universal Multi-Modal Retrieval (2024.acl-long)

Copied to clipboard

Junjie Zhou, Zheng Liu, Shitao Xiao, Bo Zhao, Yongping Xiong

MAESTRO: Meta-learning Adaptive Estimation of Scalarization Trade-offs for Reward Optimization (2026.acl-long)

Challenge:	Existing methods for multimodal retrieval are mostly text-oriented, which lack the capability to process visual information.
Approach:	They propose a multi-modal multi-text embedding model VISTA which extends a powerful text encoder with the image understanding capability by introducing visual token embedds.
Outcome:	The proposed model achieves superior performance across a variety of multi-modal retrieval tasks in zero-shot and supervised settings.

Copied to clipboard

Yang Zhao, Hepeng Wang, Xiao Ding, Yangou Ouyang, Bibo Cai, Kai Xiong, Jinglong Gao, Zhouhao Sun, Li Du, Bing Qin, Ting Liu

Green & Sustainable NLP

Challenge:	Group-Relative Policy Optimization (GRPO) has emerged as an efficient paradigm for aligning Large Language Models (LLMs), but its efficacy is confined to domains with verifiable ground truths.
Approach:	They propose a meta-cognitive orchestration layer that treats reward scalarization as a dynamic latent policy, leveraging the model’s terminal hidden states as 'a semantic bottleneck' . Across seven benchmarks, MAESTRO consistently outperforms single-reward and static multi-objective baselines while preserving the efficiency advantages of GRPO.
Outcome:	The proposed model outperforms single-reward and static multi-objective baselines while preserving efficiency advantages.

Praetor: A Fine-Grained Generative LLM Evaluator with Instance-Level Customizable Evaluation Criteria (2025.acl-long)

Copied to clipboard

Yongqi Leng, Renren Jin, Yue Chen, Zhuowen Han, Ling Shi, Jianxiang Peng, Lei Yang, Juesi Xiao, Deyi Xiong

Challenge:	Existing evaluation methods are inadequate to evaluate large language models (LLMs).
Approach:	They propose a fine-grained generative LLM evaluator with instance-level customazable evaluation criteria that can be used to evaluate large language models.
Outcome:	The proposed model outperforms existing LLM evaluators and instruction-tuned LLMs on multiple benchmarks and sets new SOTA results.

From Curated Data to Scalable Models: Continual Pre-training of Dense and MoE Large Language Models for Tibetan (2026.acl-long)

Copied to clipboard

Lei Yang, Leiyu Pan, Bojian Xiong, Renren Jin, Shaowei Zhang, Yue Chen, Ling Shi, Jiang Zhou, Junru Wu, Zhen Wang, Jianxiang Peng, Juesi Xiao, Tianyu Dong, Zhuowen Han, Zhuo Chen, Yuqi Ren, Deyi Xiong

e-CARE: a New Dataset for Exploring Explainable Causal Reasoning (2022.acl-long)

Challenge:	Large language models have achieved remarkable success across a wide range of tasks, yet their performance remains heavily biased toward high-resource languages.
Approach:	They propose a pipeline for advancing Tibetan language modeling through multilingual continual pre-training with Tibetan, Chinese, and English.
Outcome:	The proposed model outperforms open-source and Tibetan-focused models on diverse tasks.

Copied to clipboard

Li Du, Xiao Ding, Kai Xiong, Ting Liu, Bing Qin

ThinkNote: Enhancing Knowledge Integration and Utilization of Large Language Models via Constructivist Cognition Modeling (2026.findings-eacl)

Challenge:	Existing causal reasoning models only learn to induce empirical causal patterns that are predictive to the label, while human beings seek for deep and conceptual understanding of the causality to explain the observed causal facts.
Approach:	They present a human-annotated CAusal REasoning dataset with conceptual explanations of the causality.
Outcome:	The presented dataset shows that human-annotated explanations can be useful for promoting the accuracy and stability of causal reasoning models.

Copied to clipboard

Zhipeng Xu, Zhenghao Liu, Yukun Yan, Shuo Wang, Shi Yu, Zheni Zeng, Chaojun Xiao, Zhiyuan Liu, Ge Yu, Chenyan Xiong

Natural Logic at the Core: Dynamic Rewards for Entailment Tree Generation (2025.findings-acl)

Challenge:	Large Language Models (LLMs) exhibit suboptimal behaviors and inconsistencies when exposed to unfamiliar external information, underscoring their limitations in effectively leveraging such knowledge.
Approach:	They propose a framework that enhances the external knowledge utilization of Large Language Models through a two-stage constructivist cognitive modeling process.
Outcome:	The proposed framework achieves a 10% improvement over baseline methods on various question-answering benchmarks.

Copied to clipboard

Jihao Shi, Xiao Ding, Kai Xiong, Hengwei Zhao, Bing Qin, Ting Liu

MDC-Bench: A Multidisciplinary Causal Benchmark Based on Causal Structures for Evaluating Large Language Models (2026.findings-acl)

Challenge:	Existing approaches to generating entailment trees lack logical consistency . static reward structures or intricate dependencies within multi-step reasoning are often ignored .
Approach:	They propose a method that integrates natural logic principles into reinforcement learning to guide entailment tree generation.
Outcome:	Experiments on EntailmentBank show that the proposed method improves interpretability and generalization.

Copied to clipboard

Peng Wang, Yuxiong Yan, Xiao Ding, Kai Xiong, Bibo Cai, Chao Peng, Yutai Hou, Dandan Tu, Bing Qin, Ting Liu

HighMATH: Evaluating Math Reasoning of Large Language Models in Breadth and Depth (2025.findings-emnlp)

Challenge:	Existing causal datasets focus on the commonsense domain, but LLMs perform poorly when answering complex questions.
Approach:	They propose a multidisciplinary causal evaluation benchmark to assess LLMs' knowledge and skills.
Outcome:	The proposed model improves in domain specialization, structural diversity, and task complexity.

Copied to clipboard

Yan Liu, Minghui Zhang, Bojian Xiong, Yifan Xiao, Yinong Sun, Yating Mei, Longyu Zeng, Jingchao Yang, Yang Wang, Deyi Xiong

SoftDedup: an Efficient Data Reweighting Method for Speeding Up Language Model Pre-training (2024.acl-long)

Challenge:	a gap in math models' accuracy has been widened with the development of large language models (LLMs) . a new study aims to bridge this gap by evaluating a set of high-level math reasoning models .
Approach:	They propose to evaluate large language models on existing math benchmarks to bridge this gap . they collect 5,293 problems from Chinese senior high school mathematics exams .
Outcome:	The proposed model is based on o1-like models and a high-level model.

Copied to clipboard

Nan He, Weichen Xiong, Hanwen Liu, Yi Liao, Lei Ding, Kai Zhang, Guohua Tang, Xiao Han, Yang Wei

Grammar-Based Code Representation: Is It a Worthy Pursuit for LLMs? (2025.findings-acl)

Challenge:	Current methods focus on detecting and removing duplicates, which risks the loss of valuable information and neglects the varying degrees of duplication.
Approach:	They propose a method that maintains dataset integrity while selectively reducing the sampling weight of data with high commonness.
Outcome:	The proposed method significantly improves training efficiency on deduplicated datasets and improves downstream accuracy by 1.77%.

Copied to clipboard

Qingyuan Liang, Zhao Zhang, Zeyu Sun, Zheng Lin, Qi Luo, Xiao Yueyi, Yizhou Chen, Yuqun Zhang, Haotian Zhang, Lu Zhang, Chenbin Chenbin, Yingfei Xiong

Multimodality

Challenge:	Existing research demonstrates the effectiveness of grammar-based code representations in small-scale models, showing their ability to reduce syntax errors and enhance performance.
Approach:	They develop a series of billion-scale grammar-based code representations that incorporate grammar rules into the code generation process.
Outcome:	Experiments on HumanEval and MBPP show that grammar-based representations reduce syntax errors and improve performance even in billion-scale models.

Deciphering the Impact of Pretraining Data on Large Language Models through Machine Unlearning (2024.findings-acl)

Copied to clipboard

Yang Zhao, Li Du, Xiao Ding, Kai Xiong, Zhouhao Sun, Shi Jun, Ting Liu, Bing Qin

Analyzing the Rapid Generalization of SFT via the Perspective of Attention Head Activation Patterns (2025.acl-long)

Challenge:	Existing studies have suggested that the composition of the pretraining corpus exerts a significant impact upon the performance of LLMs.
Approach:	They analyze the impact of 48 datasets from 5 major categories of pretraining data of Large Language Models and measure their impacts on LLMs using benchmarks about nine major categories.
Outcome:	The proposed analysis provides insights into the organization of data to support more efficient pretraining of Large Language Models.

Copied to clipboard

Yang Zhao, Li Du, Xiao Ding, Kai Xiong, Ting Liu, Bing Qin

Green & Sustainable NLP

Challenge:	Currently, LLMs learn in a data-driven schema while the instructions about complex tasks are both scarce and hard to collect or construct.
Approach:	They employ a gradient-based method to dissect the process that the Supervised Fine-tuning Process (SFT) adapts LLMs to downstream tasks via the perspective of attention patterns.
Outcome:	The proposed method dissects the process that the SFT process adapts LLMs to downstream tasks via the perspective of attention patterns.

ExCAR: Event Graph Knowledge Enhanced Explainable Causal Reasoning (2021.acl-long)

Copied to clipboard

Li Du, Xiao Ding, Kai Xiong, Ting Liu, Bing Qin

Knowledge Representation

Challenge:	Existing work infers the causation between events based on knowledge from annotated causal event pairs, but additional evidence information is unexploited.
Approach:	They propose an Event graph knowledge enhanced explainable CAusal Reasoning framework that acquires additional evidence information from a large-scale causal event graph as logical rules for causal reasoning.
Outcome:	The proposed framework outperforms state-of-the-art methods in human evaluation and in animal models.

Com2 : A Causal-Guided Benchmark for Exploring Complex Commonsense Reasoning in Large Language Models (2025.acl-long)

Copied to clipboard

Kai Xiong, Xiao Ding, Yixin Cao, Yuxiong Yan, Li Du, Yufei Zhang, Jinglong Gao, Jiaqian Liu, Bing Qin, Ting Liu

StepCoder: Improving Code Generation with Reinforcement Learning from Compiler Feedback (2024.acl-long)

Challenge:	Existing works focus on complex tasks like math and code, while complex commonsense reasoning remains underexplored due to its uncertainty and lack of structure.
Approach:	They propose to build a benchmark for large language models based on complex commonsense reasoning based upon causal event graphs and causal theory.
Outcome:	The proposed benchmark combines a complex commonsense reasoning benchmark with a detective story to achieve a more challenging subset.

Copied to clipboard

Shihan Dou, Yan Liu, Haoxiang Jia, Enyu Zhou, Limao Xiong, Junjie Shan, Caishuang Huang, Xiao Wang, Xiaoran Fan, Zhiheng Xi, Yuhao Zhou, Tao Ji, Rui Zheng, Qi Zhang, Tao Gui, Xuanjing Huang

Programming Languages in NLP

Challenge:	Existing work integrates reinforcement learning with compiler feedback to enhance code generation quality but the long code generated by LLMs makes RL exploration ineffective.
Approach:	They propose a framework that integrates reinforcement learning and compiler feedback to enhance code generation quality.
Outcome:	The proposed framework outperforms state-of-the-art approaches in corresponding benchmarks and integrates reinforcement learning with compiler feedback to improve code generation quality.

MedCite: Can Language Models Generate Verifiable Text for Medicine? (2025.findings-acl)

Copied to clipboard

Xiao Wang, Mengjue Tan, Qiao Jin, Guangzhi Xiong, Yu Hu, Aidong Zhang, Zhiyong Lu, Minjia Zhang

OS Agents: A Survey on MLLM-based Agents for Computer, Phone and Browser Use (2025.acl-long)

Challenge:	Existing LLM-based medical question answering systems lack citation generation and evaluation capabilities, raising concerns about their adoption in practice.
Approach:	They propose a framework that facilitates the design and evaluation of LLM citations for medical tasks and a retrieval-citation method that generates high-quality citation.
Outcome:	The proposed method achieves superior citation precision and recall improvements compared to strong baseline methods and correlates well with annotation results from professional experts.

Copied to clipboard

Xueyu Hu, Tao Xiong, Biao Yi, Zishu Wei, Ruixuan Xiao, Yurun Chen, Jiasheng Ye, Meiling Tao, Xiangxin Zhou, Ziyu Zhao, Yuhuai Li, Shengze Xu, Shenzhi Wang, Xinchen Xu, Shuofei Qiao, Zhaokai Wang, Kun Kuang, Tieyong Zeng, Liang Wang, Jiwei Li, Yuchen Eleanor Jiang, Wangchunshu Zhou, Guoyin Wang, Keting Yin, Zhou Zhao, Hongxia Yang, Fan Wu, Shengyu Zhang, Fei Wu

Find Your Optimal Teacher: Personalized Data Synthesis via Router-Guided Multi-Teacher Distillation (2026.acl-long)

Challenge:	a new generation of (M)LLMs is enabling the creation of superintelligent AI assistants . OS Agents can complete tasks autonomously and have the potential to significantly enhance the lives of billions of users worldwide.
Approach:	They propose to build OS Agents that operate within operating systems' GUIs and GUIs . they examine evaluation metrics and benchmarks to identify promising directions .
Outcome:	The proposed agents are based on operating systems (OS) and operating systems frameworks.

Copied to clipboard

Hengyuan Zhang, Shiping Yang, Xiao Liang, Chenming Shang, Yuxuan Jiang, Chaofan Tao, Jing Xiong, Hayden Kwok-Hay So, Ruobing Xie, Angel X. Chang, Ngai Wong

Explaining Length Bias in LLM-Based Preference Evaluations (2025.findings-emnlp)

Challenge:	Existing studies show that stronger models are not always optimal teachers, suggesting a mismatch between the teacher’s output and the student’s learning ability.
Approach:	They propose a method that routes each prompt to its optimal teacher via a query-level router that jointly considers the student models’ learnability and teacher models’ response quality.
Outcome:	The proposed method outperforms baselines on six benchmarks including instruct tuning and math reasoning settings.

Copied to clipboard

Zhengyu Hu, Linxin Song, Jieyu Zhang, Zheyuan Xiao, Tianfu Wang, Zhengyu Chen, Nicholas Jing Yuan, Jianxun Lian, Kaize Ding, Hui Xiong

Nash CoT: Multi-Path Inference with Preference Equilibrium (2024.emnlp-main)

Challenge:	a preference evaluation metric is often biased towards longer responses, revealing a reliability problem . a decomposition of the preference evaluation into two components is needed to understand this bias.
Approach:	They propose to decompose the preference evaluation metric into two key components . the first component is length-dependent and related to trustworthiness .
Outcome:	The proposed evaluation metric is based on two components: desirability and information mass.

Copied to clipboard

Ziqi Zhang, Cunxiang Wang, Xiao Xiong, Yue Zhang, Donglin Wang