Papers by Chao Xue
Getting More from Less: Large Language Models are Good Spontaneous Multilingual Learners (2024.emnlp-main)
Copied to clipboard
Shimao Zhang, Changjiang Gao, Wenhao Zhu, Jiajun Chen, Xin Huang, Xue Han, Junlan Feng, Chao Deng, Shujian Huang
| Challenge: | Large Language Models (LLMs) have shown impressive language capabilities, but most of them have very unbalanced performance across different languages. |
| Approach: | They propose to use question translation data to enhance LLMs' multilingual capabilities by using mechanistic interpretability methods. |
| Outcome: | The proposed method improves multilingual alignment even with unannotated answers in English and a wide range of languages even with instruction-tuned LLMs. |
Beyond Token Length: Step Pruner for Efficient and Accurate Reasoning in Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing reinforcement learning methods for large reasoning models suffer from excessive verbosity, known as "overthinking." Existing models penalize generated tokens to promote conciseness, but these methods encounter two challenges: they may develop hacking behavior in later stages of training by discarding reasoning steps. |
| Approach: | They propose a framework that steers large reasoning models toward more efficient reasoning . they prioritize correctness while imposing penalties for redundant steps . |
| Outcome: | The proposed framework reduces token usage by 69.7% on AIME24. |
Log-FGAER: Logic-Guided Fine-Grained Address Entity Recognition from Multi-Turn Spoken Dialogue (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing name entity recognition methods combine pre-trained language models with supervised models such as BiLSTM/LSTM-CRF to perform poorly in a spoken dialogue context. |
| Approach: | They propose a logic-guided fine-grained address recognition method that softly applies the logic rule to improve the accuracy of FGAER. |
| Outcome: | The proposed method improves fine-grained address entity recognition from multi-turn spoken dialogues. |
Parameter Importance is Not Static: Evolving Parameter Isolation for Supervised Fine-Tuning (2026.acl-long)
Copied to clipboard
Zekai Lin, Chao Xue, Di Liang, Xingsheng Han, Peiyang Liu, Xianjie Wu, Lei Jiang, Yu Lu, Bob Simons, Shuang Liang, Minlong Peng
| Challenge: | Recent approaches to fine-tuning of large language models suffer from task interference and catastrophic forgetting. |
| Approach: | They propose a fine-tuning framework that adapts isolation decisions based on online estimates of parameter importance. |
| Outcome: | The proposed framework reduces interference and forgetting while releasing outdated parameters to recover plasticity. |
Why Supervised Fine-Tuning Fails to Learn: A Systematic Study of Incomplete Learning in Large Language Models (2026.acl-long)
Copied to clipboard
Chao Xue, Yao Wang, Mengqiao Liu, Di Liang, Xingsheng Han, Peiyang Liu, Xianjie Wu, Chenyao Lu, Lei Jiang, Yu Lu, Haibo Shi, Shuang Liang, Minlong Peng, Flora D. Salim
| Challenge: | Incomplete learning is widespread and heterogeneous in large language models . authors identify five recurrent sources of incomplete learning: missing prerequisite knowledge, conflicts between SFT supervision and pre-training knowledge, internal inconsistencies within SFT data, left-side forgetting during sequential fine-tuning, and insufficient optimization for rare or complex patterns. |
| Approach: | They propose a diagnostic-first framework that maps incomplete learning to causes . they identify five recurrent sources of incomplete learning: missing prerequisite knowledge, conflicts between supervision and pre-training knowledge, internal inconsistencies, left-side forgetting during sequential fine-tuning, and insufficient optimization for rare or complex patterns. |
| Outcome: | The proposed framework maps incomplete learning to causes using observable training and inference signals. |
Universally Empowering Zeroth-Order Optimization via Adaptive Layer-wise Sampling (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for fine-tuning Large Language Models are slow and lack of performance. |
| Approach: | They propose a Zeroth-Order optimization framework that uses forward passes to fine-tune Large Language Models. |
| Outcome: | The proposed framework achieves 1.7 to 3.0 wall-clock acceleration on LLaMA and OPT models. |
Investigating and Scaling up Code-Switching for Multilingual Language Model Pre-Training (2025.findings-acl)
Copied to clipboard
Zhijun Wang, Jiahuan Li, Hao Zhou, Rongxiang Weng, Jingang Wang, Xin Huang, Xue Han, Junlan Feng, Chao Deng, Shujian Huang
| Challenge: | Large language models (LLMs) exhibit remarkable multilingual capabilities despite the extreme language imbalance in the pre-training data. |
| Approach: | They investigate the existence of code-switching in the pre-training corpus and categorize it into four types within two quadrants. |
| Outcome: | The proposed approach improves performance across benchmarks and representation space. |
Reason Only When Needed: Efficient Generative Reward Modeling via Model-Internal Uncertainty (2026.findings-acl)
Copied to clipboard
Chao Xue, Yao Wang, Mengqiao Liu, Di Liang, Xingsheng Han, Peiyang Liu, Xianjie Wu, Chenyao Lu, Lei Jiang, Yu Lu, Haibo Shi, Shuang Liang, Minlong Peng, Flora D. Salim
| Challenge: | Existing approaches to generating reward models rely on voting-based mechanisms to evaluate CoT outputs. |
| Approach: | They propose an efficient generative reward modeling framework grounded in model-internal uncertainty. |
| Outcome: | The proposed framework reduces inference cost while improving answer accuracy. |
MultiPL-MoE: Multi-Programming-Lingual Extension of Large Language Models through Hybrid Mixture-of-Experts (2025.findings-emnlp)
Copied to clipboard
| Challenge: | MultiPL is a special case of multiple natural languages and requires limited computational resources to generate multilingual code. |
| Approach: | They propose to extend LLMs by combining two paired experts to optimize expert selection at token and segment levels. |
| Outcome: | The proposed extension improves the performance of the base LLMs while retaining the most popular ones using limited computational resources. |
Multi-Label Few-Shot Learning for Aspect Category Detection (2021.acl-long)
Copied to clipboard
| Challenge: | Existing few-shot learning methods focus on single-label predictions, which can not work well for ACD since a sentence may contain multiple aspect categories. |
| Approach: | They propose a few-shot learning method that uses the prototypical network to learn aspects from a set of aspects. |
| Outcome: | The proposed method significantly outperforms baseline methods on three datasets. |
Large Language Models Are Cross-Lingual Knowledge-Free Reasoners (2025.naacl-long)
Copied to clipboard
| Challenge: | Large language models have demonstrated impressive reasoning capabilities across multiple languages, but the relationship between capabilities in different languages is less explored. |
| Approach: | They decompose the process of reasoning tasks into two separate components: knowledge retrieval and knowledge-free reasoning. |
| Outcome: | The proposed model can be transferred across source-target languages despite secondary impact of resource in some specific target languages, while cross-lingual knowledge retrieval significantly hinders the transfer. |
Red Teaming Large Reasoning Models (2026.acl-long)
Copied to clipboard
| Challenge: | Large Reasoning Models (LRMs) have emerged as a powerful advancement in multi-step reasoning tasks, but they introduce safety and reliability risks, such as CoT-hijacking and prompt-induced inefficiencies. |
| Approach: | They propose a unified benchmark to assess the trustworthiness of Large Reasoning Models. |
| Outcome: | The proposed benchmark evaluates truthfulness, safety and efficiency on 26 models. |
LLaSE-G1: Incentivizing Generalization Capability for LLaMA-based Speech Enhancement (2025.acl-long)
Copied to clipboard
Boyi Kang, Xinfa Zhu, Zihan Zhang, Zhen Ye, Mingshuai Liu, Ziqian Wang, Yike Zhu, Guobin Ma, Jun Chen, Longshuai Xiao, Chao Weng, Wei Xue, Lei Xie
| Challenge: | Recent advances in language models have demonstrated strong capabilities in semantic understanding and contextual modeling. |
| Approach: | They propose a LLaMA-based language model that incentivizes generalization capabilities for speech enhancement. |
| Outcome: | The proposed language model outperforms prior task-specific discriminative and generative models in acoustic enhancement tasks. |
RoChBert: Towards Robust BERT Fine-tuning for Chinese (2022.findings-emnlp)
Copied to clipboard
Zihan Zhang, Jinfeng Li, Ning Shi, Bo Yuan, Xiangyu Liu, Rong Zhang, Hui Xue, Donghong Sun, Chao Zhang
| Challenge: | Pre-trained language models (e.g., BERT) have been proved vulnerable to adversarial texts. |
| Approach: | They propose to fuse Chinese phonetic and glyph features into pre-trained models by using a more comprehensive adversarial graph. |
| Outcome: | The proposed framework outperforms existing methods in significant ways on a wide range of tasks while remaining accurate on benign texts. |
Understanding LLMs’ Cross-Lingual Context Retrieval: How Good It Is And Where It Comes From (2025.emnlp-main)
Copied to clipboard
| Challenge: | Cross-lingual context retrieval is a fundamental aspect of cross-lingual alignment, but the performance and mechanism of it for large language models (LLMs) remains unclear. |
| Approach: | They evaluate cross-lingual context retrieval of over 40 large language models . they use cross-linguistic machine reading comprehension as a representative scenario . |
| Outcome: | The results show that open LLMs show strong cross-lingual context retrieval ability . the results also show that their oracle performances improve after training . |
Self-attention-based Graph-of-Thought for Math Problem Solving (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for generating reasoning paths in a chain structure are inefficient and non-human-like. |
| Approach: | They propose a decoding method for a chain-based LLM that constructs a thought graph simultaneously as an LLM inference and generates reasoning steps with a graph-structured self-attention mechanism. |
| Outcome: | The proposed method improves reasoning accuracy without huge computational over-expensive LLMs and avoids performance degradation issues when the LLM is too small to comprehend complex prompts. |
Joint Learning Event-Specific Probe and Argument Library with Differential Optimization for Document-Level Multi-Event Extraction (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for document-level multi-event extraction neglect the fine-grained difference between events in multi-documents, which leads to event confusion and missing. |
| Approach: | They propose an event-specific probe-based method to sniff multiple events by querying each corresponding argument library. |
| Outcome: | The proposed method outperforms the state-of-the-art method in the recall of multi-events. |
Beyond Layout Embedding: Layout Attention with Gaussian Biases for Structured Document Understanding (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for encoding layout information rely on millions of learnable parameters . polar coordinates provide superior choice for layout modeling, study finds . |
| Approach: | They propose to model layout attention with Gaussian biases by feeding polar coordinates into 2-D Gausssian kernels. |
| Outcome: | The proposed model improves on three widely used benchmarks. |