Papers by Zihao Xu
Feature Extraction and Steering for Enhanced Chain-of-Thought Reasoning in Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) can solve reasoning and mathematical problems using the Chain-of-Thought technique, but require costly and long CoT data and fine-tuning. |
| Approach: | They propose a method that uses Sparse Autoencoders to extract interpretable features from vanilla CoT and use them to steer the LLM's internal states. |
| Outcome: | The proposed method uses Sparse Autoencoders (SAEs) to extract interpretable features from vanilla CoT and steer the LLM's internal states during generation. |
Too Long, Do Re-weighting for Efficient LLM Reasoning Compression (2026.acl-long)
Copied to clipboard
Zhong-Zhi Li, Xiao Liang, Zihao Tang, Lei Ji, Peijie Wang, Haotian Xu, Xing W, Haizhen Huang, Weiwei Deng, Yeyun Gong, Zhijiang Guo, Xiao Liu, Fei Yin, Cheng-Lin Liu
| Challenge: | Large Language Models (LLMs) have recently achieved remarkable progress on complex reasoning tasks by leveraging extended Chain-of-Thought (CoT) techniques. |
| Approach: | They propose a method that uses Extended Chain-of-Thought (EFT) to reduce the number of output tokens by nearly 40% while maintaining the accuracy of the reasoning. |
| Outcome: | The proposed method reduces the number of output tokens by nearly 40% while maintaining the accuracy of the reasoning. |
POLYCHARTQA: Benchmarking Large Vision-Language Models with Multilingual Chart Question Answering (2026.acl-long)
Copied to clipboard
| Challenge: | Existing chart understanding benchmarks are overwhelmingly English-centric, limiting their accessibility and relevance to global audiences. |
| Approach: | They propose a multilingual chart question answering benchmark that enables efficient multilingual generation via data translation and code reuse. |
| Outcome: | The proposed benchmark systematically evaluates multilingual chart understanding on state-of-the-art LVLMs and shows a significant performance gap between English and other languages. |
Enabling Stroke-Level Structural Analysis of Hieroglyphic Scripts without Language-Specific Priors (2026.findings-acl)
Copied to clipboard
Fuwen Luo, Zihao Wan, Ziyue Wang, Yaluo Liu, Pau Tong Lin Xu, Xuanjia Qiao, Xiaolong Wang, Peng Li, Yang Liu
| Challenge: | Existing structural analysis methods for hieroglyphic scripts are script-specific and labor-intensive. |
| Approach: | They propose a hieroglyphic Stroke Analyzer framework that captures character-internal structures and semantics without handcrafted data. |
| Outcome: | The proposed framework captures character-internal structures and semantics without priors . it can be used to generalize hieroglyphic scripts across languages . |
Towards Explainable Temporal Reasoning in Large Language Models: A Structure-Aware Generative Framework (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on temporal reasoning models neglect the explainable reasoning processes underlying the results. |
| Approach: | They propose a structure-aware generative framework that integrates Graph structures with text for Explainable TEmporal Reasoning. |
| Outcome: | The proposed framework achieves state-of-the-art performance while also demonstrating robust generalization capabilities. |
SeaD: End-to-end Text-to-SQL Generation with Schema-aware Denoising (2022.findings-naacl)
Copied to clipboard
| Challenge: | Using sketch-based slot filling, text-to-SQL models suffer from over-complexity . et al., e.al., and d.albert, dr., propose a novel method for text- to-Sql generation . |
| Approach: | They propose to train sequence-to-sequence model with Schema-aware Denoising . they propose a clause-sensitive execution guided (EG) decoding strategy . |
| Outcome: | The proposed method improves performance in schema linking and grammar correctness . it also establishes new state-of-the-art on the WikiSQL benchmark . |
Generate-on-Graph: Treat LLM as both Agent and KG for Incomplete Knowledge Graph Question Answering (2024.emnlp-main)
Copied to clipboard
Yao Xu, Shizhu He, Jiabei Chen, Zihao Wang, Yangqiu Song, Hanghang Tong, Guang Liu, Jun Zhao, Kang Liu
| Challenge: | Existing methods to integrate LLMs with Knowledge Graphs (KGs) however, these methods are often incomplete to cover all the knowledge required to answer questions. |
| Approach: | They propose to integrate LLMs with Knowledge Graphs (KGs) to address insufficient knowledge and hallucination issues in Large Language Models. |
| Outcome: | The proposed method outperforms existing methods on two datasets. |
MMedAgent: Learning to Use Medical Tools with Multi-modal Agent (2024.findings-emnlp)
Copied to clipboard
Binxu Li, Tiankai Yan, Yuanting Pan, Jie Luo, Ruiyang Ji, Jiayuan Ding, Zhe Xu, Shilong Liu, Haoyu Dong, Zihao Lin, Yixin Wang
| Challenge: | Multi-modal Large Language Models (MLLMs) exhibit limited generality and often fall short when compared to specialized models. |
| Approach: | They propose a multi-modal medical agent that picks the most suitable medical tools based on user inputs. |
| Outcome: | The proposed agent performs better than open-source models and the closed-source model, GPT-4o. |
Deja vu: Contrastive Historical Modeling with Prefix-tuning for Temporal Knowledge Graph Reasoning (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing text-based methods for Temporal Knowledge Graph Reasoning struggle to balance textual knowledge and temporal information with expensive purpose-built training strategies. |
| Approach: | They propose a Contrastive historical modeling framework with prefix-tuning for TEmporal Reasoning that feeds history-contextualized text into the pseudo-Siamese encoders to strike a textual-temporal balance. |
| Outcome: | The proposed framework achieves superior performance on four transductive and three few-shot inductive TKGR benchmarks. |
Data Contamination Can Cross Language Barriers (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to detect contamination of public benchmarks are too superficial to reflect deeper forms of contamination. |
| Approach: | They propose generalization-based approaches to unmask a cross-lingual form of contamination that inflates LLMs’ performance while evading current detection methods. |
| Outcome: | The proposed model outperforms existing detection methods while avoiding contamination of public benchmarks in the pre-training data. |
Dynamic Prefix as Instructor for Incremental Named Entity Recognition: A Unified Seq2Seq Generation Framework (2025.findings-acl)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is a fundamental problem in information extraction. |
| Approach: | They propose a parameter-efficient method for Incremental Named Entity Recognition (INER) task aimed at updating a model to extract entities from an expanding set of entity type candidates by employing a dynamic prefix as a task instructor to guide the generative model. |
| Outcome: | Empirical results show that the proposed method preserves task-invariant knowledge while adapting to new entities with minimal parameter updates. |
Reinforcement Learning on Pre-Training Data (2026.acl-long)
Copied to clipboard
Siheng Li, Kejiao Li, Zenan Xu, Guanhua Huang, Kun Li, Haoyuan Wu, null Wujiajia, Zihao Zheng, Chenchen Zhang, Kun Shi, Xue Gong, Qi Yi, Ruibin Xiong, Tingqiang Xu, Yuhao Jiang, Jianfeng Yan, Yuyuan Zeng, Guanghui Xu, Jinbao Xue, Zhijiang xu, Zheng Fang, Shuai LI, Qibin Liu, Xiaoxue Li, Zhuoyu Li, Yangyu Tao, Fei Gao, Cheng Jiang, Bochao Wang, Kai Liu, Jianchen Zhu, Wai Lam, Bo Zhou, Di Wang
| Challenge: | Recent progress in large language models is driven by scaling of training compute through pre-training with nexttoken prediction (NTP) or post-training (RL) Pre-training using NTP enables models to acquire extensive knowledge and skills from general data, but it suffers from data inefficiency and catastrophic forgetting in continual learning settings. |
| Approach: | They propose to scale training compute through pre-training with next-token prediction (NTP) or post-training by scaling reinforcement learning (RL) to improve learning from general data. |
| Outcome: | Experiments on multiple benchmarks and models show that the proposed approach improves continual pre-training and provides a strong foundation for post-training on Qwen3-8B-Base. |
A Comprehensive Study of Jailbreak Attack versus Defense for Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated capabilities for generating content that could be deemed harmful. |
| Approach: | They conduct a comprehensive analysis of existing studies on jailbreaking LLMs and their defense techniques. |
| Outcome: | The proposed techniques underperform existing white-box attacks and include special tokens significantly affects the likelihood of successful attacks. |
Model Unlearning via Sparse Autoencoder Subspace Guided Projections (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing unlearning strategies lack interpretability or fail to provide robust defense against adversarial prompts. |
| Approach: | They propose a framework that leverages SAE features to drive targeted updates in the model’s parameter space. |
| Outcome: | The proposed framework reduces harmful knowledge accuracy by 3.22% compared to baselines and improves adversarial robustness under jailbreak prompts. |
Unilaw-R1: A Large Language Model for Legal Reasoning with Reinforcement Learning and Iterative Inference (2025.emnlp-main)
Copied to clipboard
| Challenge: | Reasoning-focused large language models (LLMs) are rapidly evolving across various domains, yet their capabilities in handling complex legal problems remain underexplored. |
| Approach: | They propose a large language model tailored for legal reasoning with a 7-billion parameter scale and a two-stage training strategy combining Supervised Fine-Tuning and Reinforcement Learning. |
| Outcome: | The proposed model outperforms all models of similar scale on authoritative benchmarks and outperformed Qwen-2.5-7B-Instruct (46.6%) by an average margin of 6.6%. |
Word-level Cross-lingual Structure in Large Language Models (2025.coling-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated exceptional performance across a broad spectrum of cross-lingual Natural Language Processing (NLP) tasks. |
| Approach: | They propose to use Word-level Cross-lingual Structure to prove that the word-level embedding on the hidden layers isomorphic between languages. |
| Outcome: | The proposed method significantly improves on two representative LLM foundations, LLaMA2 and BLOOM. |
SegTune: Structured and Fine-Grained Control for Song Generation (2026.acl-long)
Copied to clipboard
Yuejiao Wang, Zihao Ji, Pengfei Cai, Xu Li, Haorui Zheng, Zewen Song, Zhongliang Liu, Chen Zhang, Pengfei Wan
| Challenge: | Recent advances in neural song generation have enabled high-quality synthesis from lyrics and global textual prompts. |
| Approach: | They propose a framework that allows users to specify local musical descriptions aligned to song segments. |
| Outcome: | The proposed framework outperforms baselines in musicality and controllability. |
Fine-grained Artificial Neurons in Audio-transformers for Disentangling Neural Auditory Encoding (2023.findings-acl)
Copied to clipboard
Mengyue Zhou, Xu Liu, David Liu, Zihao Wu, Zhengliang Liu, Lin Zhao, Dajiang Zhu, Lei Guo, Junwei Han, Tianming Liu, Xintao Hu
| Challenge: | Existing studies treat each transformer encoding layer as a single artificial neuron . layer-level embeddings aggregate multiple types of contextual attention captured by multiple head modules . |
| Approach: | They propose to embed each transformer encoding layer as a single artificial neuron . they propose to couple those ANs with their biological-neuron counterparts in the human brain . |
| Outcome: | The proposed models can be used to link representations to brain activity, the authors say . their results show that the proposed models carry meaningful neurolinguistic information . |
Holistic Evaluation for Interleaved Text-and-Image Generation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing evaluation benchmarks do not support arbitrarily interleaved images and text for both inputs and outputs. |
| Approach: | They propose to use a benchmark to evaluate interleaved text-and-image generation . they define five evaluation aspects for InterleavatedEval, a reference-free metric . |
| Outcome: | The proposed benchmarks cover a limited number of domains and use cases and lack comparableity-based metrics. |
Attention Basin: Why Contextual Position Matters in Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are sensitive to the contextual position of information in input. |
| Approach: | They introduce Attention-Driven Reranking (AttnRank) which estimates a model’s intrinsic positional attention preferences using a small calibration set and reorders retrieved documents or few-shot examples to align the most salient content with these high-attention positions. |
| Outcome: | Experiments on multi-hop QA and few-shot in-context learning tasks show that AttnRank achieves substantial improvements across 10 large language models of varying architectures and scales, without modifying model parameters or training procedures. |
R2I-Bench: Benchmarking Reasoning-Driven Text-to-Image Generation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Reasoning is a fundamental capability underpinning text-to-image (T2I) generation. |
| Approach: | They propose a benchmark to rigorously assess reasoning-driven T2I generation. |
| Outcome: | Experiments with 16 representative T2I models show limited reasoning performance . a strong pipeline-based framework decouples reasoning and generation . |
The Evolution of Thought: Tracking LLM Overthinking via Reasoning Dynamics Analysis (2026.acl-long)
Copied to clipboard
Zihao Wei, Liang Pang, Jiahao Liu, Wenjie Shi, Jingcheng Deng, Shicheng Xu, Zenghao Duan, Jingang Wang, Fei Sun, Huawei Shen, Xueqi Cheng
| Challenge: | Explicit reasoning trajectories increase performance but often trigger overthinking . despite its importance, this study examines how each step of reasoning affects the final outcome . |
| Approach: | They propose a Reasoning Completion Point Detector that detects the RCP by monitoring rank dynamics of termination tokens. |
| Outcome: | The proposed method reduces token usage by up to 44% while preserving accuracy. |
A Relaxed Matching Procedure for Unsupervised BLI (2020.acl-main)
Copied to clipboard
| Challenge: | Recent studies have shown that unsupervised bilingual lexicon induction is even on par with supervised methods. |
| Approach: | They propose a relaxed matching procedure to find a more precise matching between two languages by aligning source and target embedding space bidirectionally. |
| Outcome: | The proposed method significantly outperforms previous unsupervised methods on standard benchmarks. |
Following the Autoregressive Nature of LLM Embeddings via Compression and Alignment (2025.emnlp-main)
Copied to clipboard
Jingcheng Deng, Zhongtao Jiang, Liang Pang, Zihao Wei, Liwei Chen, Kun Xu, Yang Song, Huawei Shen, Xueqi Cheng
| Challenge: | Experimental results demonstrate that our method significantly outperforms traditional contrastive learning approaches when using the same amount of data. |
| Approach: | They propose a new contrastive learning method built on embedding conditional probability distributions that integrates two tasks: information compression and conditional distribution alignment. |
| Outcome: | The proposed method outperforms traditional contrastive learning approaches and achieves comparable performance to state-of-the-art models when using the same amount of data. |
SLIM: Subtrajectory-Level Elimination for More Effective Reasoning (2025.findings-emnlp)
Copied to clipboard
Xifeng Yao, Chengyuan Ma, Dongyu Lang, Yinhao Ni, Zhiwei Xu, Huarui Xie, Zihao Chen, Guang Shen, Dandan Tu, Yi Bai, Changzheng Zhang
| Challenge: | Notable examples include OpenAI’s o1/o3/o4 series and DeepSeek-R1 . |
| Approach: | They develop a framework to identify suboptimal subtrajectories based on human-established criteria . they also use a sampling algorithm to select data whose reasoning process is free from suboptimally subtravertories to the highest degree . |
| Outcome: | The proposed method reduces the number of suboptimal subtrajectories by 25.9% during the inference process. |
GlotEval: A Test Suite for Massively Multilingual Evaluation of Large Language Models (2025.emnlp-demos)
Copied to clipboard
Hengyu Luo, Zihao Li, Joseph Attieh, Sawal Devkota, Ona de Gibert, Xu Huang, Shaoxiong Ji, Peiqin Lin, Bhavani Sai Praneeth Varma Mantina, Ananda Sreenidhi, Raúl Vázquez, Mengjie Wang, Samea Yusofi, Fei Yuan, Jörg Tiedemann
| Challenge: | Existing evaluation frameworks focus on English and a handful of high-resource languages, thereby overlooking the realistic performance of large language models in multilingual and lower-resourced scenarios. |
| Approach: | They propose a unified and lightweight framework that integrates 27 benchmarks under a standard ISO 639-3 language identifier system to enable seamless incorporation of new benchmarks. |
| Outcome: | The proposed framework integrates 27 benchmarks under a standard ISO 639-3 language identifier system, allowing for seamless incorporation of new benchmarks. |
Token-level Preference Self-Alignment Optimization for Multi-style Outline Controllable Generation (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing attempts to outline generation are limited by response pair requirements and substantial computation costs. |
| Approach: | They propose a token-level preference self-alignment optimization for outline controllable generation that extends the Bradley-Terry model from pair-wise to list-wise comparison. |
| Outcome: | The proposed method outperforms existing methods by 19.28% in performance while requiring only 56.25% training time. |
Semi-Supervised Bilingual Lexicon Induction with Two-way Interaction (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing semisupervised methods do not fully utilize the knowledge hidden in annotated and nonannotated data, which hinders further improvement of their performance. |
| Approach: | They propose a semi-supervised BLI framework to encourage interaction between supervised signal and unsupervised alignment. |
| Outcome: | The proposed framework can incorporate any supervised and unsupervised BLI methods based on optimal transport and bi-directional lexicon update. |