Papers by Xue He
SwiftPrune: Hessian-Free Weight Pruning for Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a novel post-training pruning method relies on the Hessian matrix to perform pruning . current pruning methods are computationally intensive and lack performance due to second-order derivative calculations. |
| Approach: | They propose a Hessian-free weight pruning method that reduces computational burden . they use an Exponentially Weighted Moving Average technique to bypass weight sorting . |
| Outcome: | The proposed method achieves hardware-efficient model compression by eliminating computational intensive calculations. |
Beyond Token Length: Step Pruner for Efficient and Accurate Reasoning in Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing reinforcement learning methods for large reasoning models suffer from excessive verbosity, known as "overthinking." Existing models penalize generated tokens to promote conciseness, but these methods encounter two challenges: they may develop hacking behavior in later stages of training by discarding reasoning steps. |
| Approach: | They propose a framework that steers large reasoning models toward more efficient reasoning . they prioritize correctness while imposing penalties for redundant steps . |
| Outcome: | The proposed framework reduces token usage by 69.7% on AIME24. |
KEHRL: Learning Knowledge-Enhanced Language Representations with Hierarchical Reinforcement Learning (2024.lrec-main)
Copied to clipboard
| Challenge: | General pre-trained language models (PLMs) leverage relation triples from knowledge graphs (KGs) and integrate external data sources into language models via self-supervised learning. |
| Approach: | They propose to learn Knowledge-Enhanced language representations with Hierarchical Reinforcement Learning (KEHRL) to detect positions for knowledge injection and integrate external knowledge into the model to avoid injecting inaccurate or irrelevant knowledge. |
| Outcome: | The proposed model can detect essential positions in texts for knowledge injection and integrate external knowledge into the model to avoid injecting inaccurate or irrelevant knowledge. |
MMRC: A Large-Scale Benchmark for Understanding Multimodal Large Language Model in Real-World Conversation (2025.acl-long)
Copied to clipboard
Haochen Xue, Feilong Tang, Ming Hu, Yexin Liu, Qidong Huang, Yulong Li, Chengzhi Liu, Zhongxing Xu, Chong Zhang, Chun-Mei Feng, Yutong Xie, Imran Razzak, Zongyuan Ge, Jionglong Su, Junjun He, Yu Qiao
| Challenge: | Existing multimodal large language models lack the ability to memorize, recall, and reason in sustained interactions. |
| Approach: | They propose a multimodal real-world conversation benchmark for evaluating open-ended abilities of multimodal large language models. |
| Outcome: | The proposed benchmarks show that the models perform better in open-ended conversations. |
Sparse Black-Box Multimodal Attack for Vision-Language Adversary Generation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing adversarial attacks using imperceptible perturbations are challenging to simulate . e-commerce product restrictions and hate speech monitoring are examples of such attacks . |
| Approach: | They propose a black-box adversarial attack that leverages sparse perturbations to simulate adversarials exhibited by illegal merchants in the black- box scenario. |
| Outcome: | The proposed method outperforms existing attacks and unimodal attacks by treating images and text in discrete space and outperforming existing models. |
CogKGE: A Knowledge Graph Embedding Toolkit and Benchmark for Representing Multi-source and Heterogeneous Knowledge (2022.acl-demo)
Copied to clipboard
Zhuoran Jin, Tianyi Men, Hongbang Yuan, Zhitao He, Dianbo Sui, Chenhao Wang, Zhipeng Xue, Yubo Chen, Jun Zhao
| Challenge: | Existing methods focus on entity-centric knowledge, but CogKGE supports heterogeneous knowledge. |
| Approach: | They propose a knowledge graph embedding toolkit to represent multi-source and heterogeneous knowledge. |
| Outcome: | The proposed toolkit provides a unified programming framework for KGE tasks and a series of knowledge representations for downstream tasks. |
TRELM: Towards Robust and Efficient Pre-training for Knowledge-Enhanced Language Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for incorporating external knowledge into language models do not prioritize learning embeddings for entity-related tokens. |
| Approach: | They propose a framework for incorporating external knowledge into pre-training models that utilize entity-related tokens. |
| Outcome: | The proposed framework reduces pre-training time by 50% and outperforms other KEPLMs in knowledge probing tasks and multiple knowledge-aware language understanding tasks. |
Rationales for Answers to Simple Math Word Problems Confuse Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent studies show that large language models have advanced mathematical problem-solving abilities in grade school math word problems. |
| Approach: | They propose to combine fine-tuning and prompt-based methods to improve performance . they propose to use a hybrid algorithm to fine- tune LLMs on specific tasks . |
| Outcome: | The proposed methods improve performance on the proposed reasoning process evaluation benchmarks. |
SpanMlt: A Span-based Multi-Task Learning Framework for Pair-wise Aspect and Opinion Terms Extraction (2020.acl-main)
Copied to clipboard
| Challenge: | Aspect terms and opinion terms are key problems of fine-grained aspect-based sentiment analysis. |
| Approach: | They propose a method to extract aspect and opinion terms as pairs from a sentence . they use shared spans to extract the terms under supervision of span boundaries . |
| Outcome: | The proposed method outperforms state-of-the-art methods on both aspects and opinion terms extraction tasks. |
Attention Consistency for LLMs Explanation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing interpretability methods face limitations such as low resolution and high computational cost. |
| Approach: | They propose a multi-layer attention consistency score to estimate the importance of input tokens in large language models. |
| Outcome: | The proposed heuristic achieves a favorable trade-off between interpretability quality and computational efficiency . |
Don’t Act Blindly: Robust GUI Automation via Action-Effect Verification and Self-Correction (2026.acl-long)
Copied to clipboard
Yuzhe Zhang, Xianwei Xue, Xingyong Wu, Mengke Chen, Chen Liu, Xinran He, Run Shao, Feiran Liu, Huanmin Xu, Qiutong Pan, Haiwei Wang
| Challenge: | Existing GUI agents assume deterministic environment responses, generating actions without verifying whether previous operations succeeded. |
| Approach: | They propose a GUI agent that explicitly models action outcomes and recovery under noisy environments. |
| Outcome: | The proposed agent reduces failure loops and improves recovery success in noisy environments while maintaining competitive standard task performance. |
FinReporting: An Agentic Workflow for Localized Reporting of Cross-Jurisdiction Financial Disclosure (2026.acl-demo)
Copied to clipboard
Fan Zhang, Mingzi Song, Rania Elbadry, Yankai Chen, Shaobo Wang, Yixi Zhou, Xunwen Zheng, Yueru He, Yuyang Dai, Georgi Nenkov Georgiev, Ayesha Gull, Muhammad Usman Safder, Fan Wu, Liyuan Meng, Fengxian Ji, Junning Zhao, Xueqing Peng, Jimin Huang, YU Chen, Xue Liu, Preslav Nakov, Zhuohan Xie
| Challenge: | FinReporting is an agentic workflow for localized cross-jurisdiction financial reporting . existing approaches assume a single-market setting and overlook structural differences across jurisdictions . |
| Approach: | They propose a workflow that decomposes financial reporting into auditable stages . they use Large Language Models to extract and summarize corporate disclosures . |
| Outcome: | The proposed system decomposes reporting into auditable stages . it improves consistency and reliability under heterogeneous reporting regimes. |
CHESS: Optimizing LLM Inference via Channel-Wise Thresholding and Selective Sparsification (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for activation sparsification do not capture the relationship between activation and model performance. |
| Approach: | They propose a general activation sparsification approach using channel-wise thresholding and selective sparsifying to capture the relationship between activation and model performance. |
| Outcome: | The proposed approach reduces the number of activated neurons during inference by 1.27x over eight downstream tasks while activating fewer parameters than existing methods. |
Safe-SAIL: Towards a Fine-grained Safety Landscape of Large Language Models via Sparse Autoencoder Interpretation Framework (2026.findings-acl)
Copied to clipboard
Jiaqi Weng, Han Zheng, Hanyu Zhang, Ej Zhou, Qinqin He, Jialing Tao, Hui Xue, Zhixuan Chu, Xiting Wang
| Challenge: | Existing studies on how SAEs derive most fine-grained latent features for safety remain unexplored. |
| Approach: | They propose a framework for interpreting SAE features in safety-critical domains . they train a suite of SAEs with human-readable explanations and systematic evaluations based on pornography, politics, violence, and terror . |
| Outcome: | The proposed framework reduces interpretation cost by 55% and improves safety-critical features. |
The Right Time Matters: Data Arrangement Affects Zero-Shot Generalization in Instruction Tuning (2025.findings-acl)
Copied to clipboard
Bingxiang He, Ning Ding, Cheng Qian, Jia Deng, Ganqu Cui, Lifan Yuan, Haiwen Hong, Huan-ang Gao, Longtao Huang, Hui Xue, Huimin Chen, Zhiyuan Liu, Maosong Sun
| Challenge: | Existing work on instruction tuning has focused on task level, without considering that tasks are artificially defined and, to LLMs, merely consist of tokens and representations. |
| Approach: | They propose a training data arrangement framework that allows for continual learning and loss reduction. |
| Outcome: | The proposed framework promotes continual learning and loss reduction on unseen tasks. |
A2ATS: Retrieval-Based KV Cache Reduction via Windowed Rotary Position Embedding and Query-Aware Vector Quantization (2025.findings-acl)
Copied to clipboard
Junhui He, Junna Xing, Nan Wang, Rui Xu, Shangyu Wu, Peng Zhou, Qiang Liu, Chun Jason Xue, Qingan Li
| Challenge: | Long context large language models (LLMs) pose significant challenges for efficient serving due to the large memory footprint and high access overhead of KV cache. |
| Approach: | They propose a retrieval-based method to reduce the memory footprint of LLMs . they propose Windowed Rotary Position Embedding and query-aware vector quantization . |
| Outcome: | The proposed method can achieve lower performance degradation with lower overhead compared to existing methods . it can reduce the memory footprint and access overhead of long context large language models . |
Select to Know: An Internal-External Knowledge Self-Selection Framework for Domain-Specific Question Answering (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) perform well in general QA but often struggle in domain-specific scenarios. |
| Approach: | They propose a framework that internalizes domain knowledge through internal-external knowledge self-selection and selective supervised fine-tuning. |
| Outcome: | The proposed framework outperforms existing methods and matches domain-pretrained LLMs with significantly lower cost. |
RISE: Reasoning Enhancement via Iterative Self-Exploration in Multi-hop Question Answering (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) excel in many areas but face challenges with complex reasoning tasks, such as Multi-Hop Question Answering (MHQA). |
| Approach: | They propose a framework to enhance models’ reasoning capability through iterative self-exploration that addresses key errors in MHQA tasks such as Evidence Aggregation and Reasoning Decomposition. |
| Outcome: | Extensive experiments on multiple MHQA benchmarks show that the proposed framework significantly improves reasoning accuracy and task performance. |
MSMO-ABSA: Multi-Scale and Multi-Objective Optimization for Cross-Lingual Aspect-Based Sentiment Analysis (2026.acl-long)
Copied to clipboard
| Challenge: | Aspect-based sentiment analysis (ABSA) has seen success with English texts, but real-world social media interactions often involve multiple languages. |
| Approach: | They propose a framework for cross-lingual ABSA that incorporates code-switched bilingual sentences into the language discriminator and consistency training modules to enhance cross-linguistic alignment. |
| Outcome: | The proposed framework achieves cross-lingual sentence-level and aspect-level alignment, aligning features of aspect terms in different contextual environments. |
DiffER: Diffusion Entity-Relation Modeling for Reversal Curse in Diffusion Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing large language models exhibit unidirectional behavior when processing bidirectional relationships . authors propose a solution to alleviate the reversal curse in Diffusion LLMs . |
| Approach: | They propose a model that addresses the "reversal curse" of bidirectional behavior in large language models . they propose 'entity-aware training' and balanced data construction to alleviate asymmetry and missing relations . |
| Outcome: | The proposed model alleviates the "reversal curse" in Diffusion LLMs . the proposed model employs whole-entity masking to mitigate entity fragmentation . |
Scaling Behaviors of LLM Reinforcement Learning Post-Training: An Empirical Study in Mathematical Reasoning (2026.acl-long)
Copied to clipboard
Zelin Tan, Hejia Geng, Xiaohang Yu, Mulei Zhang, Guancheng Wan, Yifan Zhou, Qiang He, Xiangyuan Xue, Heng Zhou, Yutao Fan, Zhong-Zhi Li, Zaibin Zhang, Guibin Zhang, Chen Zhang, Zhenfei Yin, Philip Torr, Lei Bai
| Challenge: | elucidating scaling laws for large language models (LLMs) during pre-training remains unexplored. |
| Approach: | They characterize how model scale, data, and compute interact during pre-training . they find that large models consistently demonstrate superior compute and data efficiency . |
| Outcome: | The proposed scaling laws offer practical guidance for scaling reasoning capabilities through reinforcement learning post-training. |
Graph-Based Chain-of-Thought Pruning for Reducing Redundant Reflections in Reasoning LLMs (2026.findings-acl)
Copied to clipboard
Hongyuan Yuan, Xinran He, Run Shao, Bolei He, Xianwei Xue, Mengke Chen, Qiutong Pan, Haiwei Wang, Haifeng Li
| Challenge: | Extending CoT through RL can induce undesirable thinking patterns such as overthinking . prior work has focused on inefficient reflection, which manifests in two problematic patterns: Indiscriminate Reflection and Repetitive Reflectione . |
| Approach: | They propose a graph-based approach to optimize CoT by pruning each linear CoT into a directed acyclic graph with explicit dependency edges. |
| Outcome: | The proposed approach reduces the average reasoning tokens by 42% while maintaining or improving accuracy. |
Gradient-Adaptive Policy Optimization: Towards Multi-Objective Alignment of Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Reinforcement learning from human feedback (RLHF) has emerged as a powerful technique for aligning large language models (LLMs) with human preferences. |
| Approach: | They propose a novel algorithm that uses multiple-gradient descent to optimize LLMs with diverse preferences to maximize trade-offs between objectives. |
| Outcome: | The proposed approach incorporates user preferences across different objectives and achieves Pareto solutions that better align with the user’s specific needs. |
MLWQ: Efficient Small Language Model Deployment via Multi-Level Weight Quantization (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for efficient deployment of small language models face inefficient bit-width allocation and insufficient fine-grained quantization adjustments. |
| Approach: | They propose a weight quantization technique that facilitates efficient deployment of SLMs . they propose to combine inter-layer loss and intra-layer salience to achieve better allocation . |
| Outcome: | Experimental results show that multi-level weight quantization achieves competitive performance compared to state-of-the-art methods. |
UniPSDA: Unsupervised Pseudo Semantic Data Augmentation for Zero-Shot Cross-Lingual Natural Language Understanding (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing studies rely on shallow unsupervised data generated by token surface matching regardless of global context-aware semantics of the surrounding text tokens. |
| Approach: | They propose an Unsupervised Pseudo Semantic Data Augmentation mechanism to enrich training data without human intervention. |
| Outcome: | The proposed model improves on general zero-shot cross-lingual understanding tasks on different languages without human intervention. |