Papers by Ao Xu

Seq1F1B: Efficient Sequence-Level Pipeline Parallelism for Large Language Model Training (2025.naacl-long)

Copied to clipboard

Sun Ao, Weilin Zhao, Xu Han, Cheng Yang, Xinrong Zhang, Zhiyuan Liu, Chuan Shi, Maosong Sun

Challenge:	Current PP methods face severe bottlenecks, including pipeline bubbles and memory footprint.
Approach:	They propose a sequence-level one-forward-one-backward (1F1B) PP method for training LLMs on long sequences with high throughput and memory efficiency.
Outcome:	The proposed method achieves 1.14X training throughput with half memory footprint compared to baseline methods . it trains an LLM with 30B parameters on sequences up to 64k tokens using 64X NVIDIA A100 GPUs .

A Challenge Dataset and Effective Models for Aspect-Based Sentiment Analysis (D19-1)

Copied to clipboard

Qingnan Jiang, Lei Chen, Ruifeng Xu, Xiang Ao, Min Yang

Sentiment Analysis

Challenge:	Existing ABSA methods only use one aspect or multiple aspects with the same sentiment polarity . recent studies show that neural network methods can be trained end-to-end and automatically learn important features.
Approach:	They propose a large-scale multi-aspect multi-sentiment dataset with two different aspects with different sentiment polarities.
Outcome:	The proposed model outperforms the state-of-the-art models on the large-scale dataset . it is based on a novel neural network approach that can be trained end-to-end .

APB: Accelerating Distributed Long-Context Inference by Passing Compressed Context Blocks across GPUs (2025.acl-long)

Copied to clipboard

Yuxiang Huang, Mingye Li, Xu Han, Chaojun Xiao, Weilin Zhao, Sun Ao, Hao Zhou, Jie Zhou, Zhiyuan Liu, Maosong Sun

Responsible & Trustworthy NLP

Challenge:	Long-context inference is crucial for advancing large language models, but its prefill speed remains a bottleneck.
Approach:	They propose an efficient long-context inference framework that leverages multi-host approximate attention to enhance prefill speed.
Outcome:	The proposed framework achieves speedups of 9.2, 4.2, and 1.6 without any degradation in performance.

Interactive Key-Value Memory-augmented Attention for Image Paragraph Captioning (2020.coling-main)

Copied to clipboard

Chunpu Xu, Yu Li, Chengming Li, Xiang Ao, Min Yang, Jinwen Tian

Multimodality

Challenge:	Existing approaches to image paragraph captioning ignore the past alignment information, resulting in repetitive captioning and incomplete captioning.
Approach:	They propose an Interactive key-value Memory-augmented Attention model for image paragraph captioning to keep track of attention history along with update-chain of decoder state.
Outcome:	Extensive experiments on a benchmark dataset demonstrate the effectiveness of the proposed model.

Meet Changes with Constancy: Learning Invariance in Multi-Source Translation (2020.coling-main)

Copied to clipboard

Jianfeng Liu, Ling Luo, Xiang Ao, Yan Song, Haoran Xu, Jian Ye

Machine Translation

Challenge:	Existing approaches to multi-source neural machine translation neglect inconsistencies between sources of information.
Approach:	They propose a source invariance network to learn invariant information of parallel sources . they propose to integrate such network with multi-encoder based multi-source NMT methods .
Outcome:	The proposed approach achieves clear gains in translation quality and captures implicit invariance between different sources.

APB-V: Accelerating Long-Video Understanding via Sequence-Parallelism-aware Approximate Attention (2026.acl-long)

Copied to clipboard

Yuxiang Huang, Mingye Li, Xu Han, Chaojun Xiao, Weilin Zhao, Ao Sun, Ziqi Yuan, Hao Zhou, Fandong Meng, Zhiyuan Liu

Semantic Text Processing

Challenge:	Existing methods for long-video inference use compression or sparse attention . existing methods restrict LMMs from handling longer, more complex videos .
Approach:	They propose a sequence-parallel framework with optimized attention that accelerates long-video inference across multiple GPUs.
Outcome:	The proposed framework delivers speedups of 12.72x, 1.70x, and 1.18x over FlashAttn, ZigZagRing, and APB without significant performance loss.

FR-Spec: Accelerating Large-Vocabulary Language Models via Frequency-Ranked Speculative Sampling (2025.acl-long)

Copied to clipboard

Weilin Zhao, Tengyu Pan, Xu Han, Yudi Zhang, Sun Ao, Yuxiang Huang, Kaihuo Zhang, Weilun Zhao, Yuxuan Li, Jie Zhou, Hao Zhou, Jianyong Wang, Maosong Sun, Zhiyuan Liu

Language Models

Challenge:	Speculative sampling is an efficient way to accelerate the auto-regressive generation process of large language models.
Approach:	They propose a frequency-ranked speculative sampling framework that optimizes draft candidate selection through vocabulary space compression.
Outcome:	Experiments show that FR-Spec reduces LM Head computation overhead by 75% while ensuring the equivalence of the final output distribution.

Good Reasoning Makes Good Demonstrations: Implicit Reasoning Quality Supervision via In-Context Reinforcement Learning (2026.findings-acl)

Copied to clipboard

Tiehua Mei, Minxuan Lv, Leiyu Pan, Zhenpeng Su, Hongru Hou, Hengrui Chen, Ao Xu, Deqing Yang

Reasoning

Challenge:	Reinforcement Learning with Verifiable Rewards (RLVR) improves reasoning in large language models but treats all correct solutions equally, potentially reinforcing flawed traces that arrive at correct answers by chance.
Approach:	They propose a method that reweights rewards by a factor approximately proportional to Evidence Gain and assigns higher weights to high-quality traces without requiring costly computation.
Outcome:	Experiments on mathematical reasoning benchmarks show that Reinforcement Learning with Verifiable Rewards (RLVR) improves reasoning in large language models but treats all correct solutions equally.

E-EVAL: A Comprehensive Chinese K-12 Education Evaluation Benchmark for Large Language Models (2024.findings-acl)

Copied to clipboard

Jinchang Hou, Chang Ao, Haihong Wu, Xiangtao Kong, Zhigang Zheng, Daijia Tang, Chengming Li, Xiping Hu, Ruifeng Xu, Shiwen Ni, Min Yang

Language Models

Challenge:	despite the rapid development of Large Language Models, there is no dedicated benchmark for evaluating LLMs in Chinese K-12 education.
Approach:	They propose to develop a benchmark specifically tailored for Chinese K-12 education.
Outcome:	EVAL is the first evaluation benchmark specifically tailored for Chinese K-12 education.

Modelling Long-distance Node Relations for KBQA with Global Dynamic Graph (2020.coling-main)

Copied to clipboard

Xu Wang, Shuai Zhao, Jiale Han, Bo Cheng, Hao Yang, Jianchang Ao, Zhenzi Li

Structured Data in NLP

Challenge:	Existing studies rely on deep graph neural networks (GNNs) to capture rich structural information, but they lack the structural information needed for QA.
Approach:	They propose a framework which captures structural information from KBs and models long-distance node relations from two perspectives.
Outcome:	The proposed framework models long-distance node relations from two perspectives . it is based on two widely used multi-hop KBQA datasets .

Gradient-Adaptive Policy Optimization: Towards Multi-Objective Alignment of Large Language Models (2025.acl-long)

Copied to clipboard

Chengao Li, Hanyu Zhang, Yunkun Xu, Hongyan Xue, Xiang Ao, Qing He

Language Models

Challenge:	Reinforcement learning from human feedback (RLHF) has emerged as a powerful technique for aligning large language models (LLMs) with human preferences.
Approach:	They propose a novel algorithm that uses multiple-gradient descent to optimize LLMs with diverse preferences to maximize trade-offs between objectives.
Outcome:	The proposed approach incorporates user preferences across different objectives and achieves Pareto solutions that better align with the user’s specific needs.