Papers by Lei He

Sequicity: Simplifying Task-oriented Dialogue Systems with Single Sequence-to-Sequence Architectures (P18-1)

Challenge:	Existing methods for numerical reasoning are not flexible enough to handle diverse expressions.
Approach:	They propose a Relational Graph enhanced Hybrid table-text Numerical reasoning model with Tree decoder which captures relationship between numerical value, table schema, and text information on the encoder side.
Outcome:	The proposed model outperforms the baseline model and achieves state-of-the-art results on the publicly available tabletext hybrid QA benchmark.

Copied to clipboard

Wenqiang Lei, Xisen Jin, Min-Yen Kan, Zhaochun Ren, Xiangnan He, Dawei Yin

Paraphrasing

Challenge:	Existing solutions to task-oriented dialogue systems follow pipeline designs which introduces complexity and fragility.
Approach:	They propose a novel sequence-to-sequence (seq2sequ) model which tracks dialogue believes and a two stage copynet instantiation which emonstrates good scalability.
Outcome:	The proposed framework outperforms state-of-the-art pipeline-based methods on large datasets and retains satisfactory entity match rate on out-of vocabulary (OOV) cases where pipeline-designed competitors totally fail.

CORD: Bridging the Audio–Text Reasoning Gap via Weighted On-policy Cross-modal Distillation (2026.findings-acl)

Copied to clipboard

Hu Jing, Danxiang Zhu, Xianlong Luo, Dan Zhang, Shuwei He, Yishu Lei, Shikun Feng, Hai-Tao Zheng, Jingzhou HE, Yu Sun, Hua Wu, Haifeng Wang

The JDDC Corpus: A Large-Scale Multi-Turn Chinese Dialogue Dataset for E-commerce Customer Service (2020.lrec-1)

Challenge:	Large Audio Language Models (LALMs) exhibit a degradation in knowledge and reasoning capabilities . empirical results show that CORD significantly bridges the audio–text performance gap .
Approach:	They propose a framework that performs online cross-modal self-distillation to bridge the acoustic-semantic gap between LALMs and text-based models.
Outcome:	The proposed framework bridges the acoustic-semantic gap between LALMs and text-based models . it employs on-policy reverse KL divergence with importance-aware weighting .

Copied to clipboard

Meng Chen, Ruixue Liu, Lei Shen, Shaozu Yuan, Jingyan Zhou, Youzheng Wu, Xiaodong He, Bowen Zhou

Natural Language Interfaces

Challenge:	Existing datasets for human-like dialogue tasks are deficient due to the complexity of human conversations.
Approach:	They construct a large-scale Chinese E-commerce conversation corpus with 1 million dialogues, 20 million utterances, and 150 million words.
Outcome:	The proposed dataset includes 1 million multi-turn dialogues, 20 million utterances, and 150 million words.

Dialog-Post: Multi-Level Self-Supervised Objectives and Hierarchical Model for Dialogue Post-Training (2023.acl-long)

Copied to clipboard

Zhenyu Zhang, Lei Shen, Yuming Zhao, Meng Chen, Xiaodong He

ESGenius: Benchmarking LLMs on Environmental, Social, and Governance (ESG) and Sustainability Knowledge (2025.emnlp-main)

Challenge:	a new method for dialogue representation and understanding is proposed . pre-trained language models (PLMs) are inappropriate for dialogue understanding tasks .
Approach:	They propose a method that trains pre-trained language models to fit dialogues . they use a hierarchical segment-wise self-attention network to model dialogues more comprehensively .
Outcome:	The proposed method outperforms existing models and achieves a 3.3% improvement on average.

Copied to clipboard

Chaoyue He, Xin Zhou, Yi Wu, Xinjia Yu, Yan Zhang, Lei Zhang, Di Wang, Shengfei Lyu, Hong Xu, Wang Xiaoqiao, Wei Liu, Chunyan Miao

On the Sentence Embeddings from Pre-trained Language Models (2020.emnlp-main)

Challenge:	ESGenius is a comprehensive benchmark for evaluating Large Language Models on ESG and sustainability knowledge.
Approach:	They introduce ESGenius, a benchmark for evaluating and enhancing ESG proficiency . they use a rigorous two-stage evaluation protocol and a repository of foundational frameworks .
Outcome:	ESGenius is a benchmark for evaluating and enhancing the proficiency of Large Language Models (LLMs) in ESG and sustainability-focused question answering.

Copied to clipboard

Bohan Li, Hao Zhou, Junxian He, Mingxuan Wang, Yiming Yang, Lei Li

Rethinking Smoothness for Fast and Adaptable Entity Alignment Decoding (2025.findings-naacl)

Challenge:	Pre-trained contextual representations like BERT have been widely used for NLP tasks.
Approach:	They propose to transform anisotropic sentence embedding distribution to smooth and isotropic Gaussian distribution by normalizing flows that are learned with an unsupervised objective.
Outcome:	The proposed method achieves significant performance gains over state-of-the-art embeddings on a variety of semantic textual similarity tasks.

Copied to clipboard

Yuanyi Wang, Han Li, Haifeng Sun, Lei Zhang, Bo He, Wei Tang, Tianhao Yan, Qi Qi, Jingyu Wang

Optimizing Question Semantic Space for Dynamic Retrieval-Augmented Multi-hop Question Answering (2025.acl-long)

Challenge:	Existing methods for integrating knowledge graphs rely on entity and relation embeddings . Fig. 1 shows how to decode knowledge graph in under 6 seconds .
Approach:	They propose a framework that only utilizes entity embeddings to decode knowledge graphs.
Outcome:	The proposed framework reconstructs KG representation by maximizing smoothness of entity embeddings.

Copied to clipboard

Linhao Ye, Lang Yu, Zhikai Lei, Qin Chen, Jie Zhou, Liang He

Information Retrieval

Challenge:	Existing retrieval-augmented generation methods are insufficient for multi-hop question answering . however, they tend to generate hallucinations due to semantic mismatching .
Approach:	They propose to optimize question semantic space for dynamic retrieval-augmented multi-hop question answering by optimizing the semantic embeddings.
Outcome:	The proposed method outperforms existing RAG methods in both in- and out-of-domain settings.

DrAgent: Empowering Large Language Models as Medical Agents for Multi-hop Medical Reasoning (2025.findings-emnlp)

Copied to clipboard

Fenglin Liu, Zheng Li, Hongjian Zhou, Qingyu Yin, Jingfeng Yang, Xin Liu, Zhengyang Wang, Xianfeng Tang, Shiyang Li, Xiang He, Ruijie Wang, Bing Yin, Xiao Gu, Lei Clifton, David A. Clifton

Uncertainty-Aware Cross-Lingual Transfer with Pseudo Partial Labels (2022.findings-naacl)

Challenge:	commercial LLMs can be difficult to use in real-world clinical decision-making . a lightweight LLM can be used to collaborate with diverse clinical tools .
Approach:	They propose a lightweight LLM that can be used to build medical LLMs as agents . they use recursive curriculum learning to optimize the LLM in an easy-to-hard progression .
Outcome:	The proposed approach outperforms human experts in medical examinations on diverse datasets.

Copied to clipboard

Shuo Lei, Xuchao Zhang, Jianfeng He, Fanglan Chen, Chang-Tien Lu

TART: Improved Few-shot Text Classification Using Task-Adaptive Reference Transformation (2023.acl-long)

Challenge:	Existing methods to train pre-trained language models for zero-shot cross-lingual tasks are noisy and lack confidence.
Approach:	They propose an uncertainty-aware cross-lingual transfer framework with pseudo-partial-label to maximize the utilization of unlabeled data by reducing noise.
Outcome:	The proposed framework outperforms baselines on named entity recognition and natural language inference tasks on 40 languages.

Copied to clipboard

Shuo Lei, Xuchao Zhang, Jianfeng He, Fanglan Chen, Chang-Tien Lu

PodAgent: A Comprehensive Framework for Podcast Generation (2025.findings-acl)

Challenge:	Existing methods for fewshot text classification depend on inter-class variance . Existing approaches suffer from MLADA, which performs poorly on tasks with high inter- class variance whereas it fails to distinguish samples from tasks with low inter-group variance.
Approach:	They propose a task-adaptive reference transformation network to transform class prototypes to per-class fixed reference points in task-adapted metric spaces.
Outcome:	The proposed method surpasses state-of-the-art methods in 1-shot and 5-shot classifications on the 20 Newsgroups dataset.

Copied to clipboard

Yujia Xiao, Lei He, Haohan Guo, Feng-Long Xie, Tan Lee

Latent Suicide Risk Detection on Microblog via Suicide-Oriented Word Embeddings and Layered Attention (D19-1)

Challenge:	Existing automatic audio generation methods struggle to generate podcast-like audio programs effectively.
Approach:	They propose a framework for creating podcast-like audio programs that generates informative topic-discussion content by designing a multi-agent collaboration system, builds a voice pool and uses LLM-enhanced speech synthesis to generate expressive conversational speech.
Outcome:	The proposed framework surpasses direct GPT-4 generation in topic-discussion dialogue content, and produces more expressive conversational speech.

Copied to clipboard

Lei Cao, Huijun Zhang, Ling Feng, Zihan Wei, Xin Wang, Ningyun Li, Xiaohao He

MinerU2.5: A Decoupled Vision-Language Model for Efficient High-Resolution Document Parsing (2026.acl-industry)

Challenge:	Existing approaches to detect suicidal ideation on social media are limited to a small group of people.
Approach:	They propose to use tree holes to embed words into microblogs to strengthen the sensibility of suicide-related lexicons and to use a two-layered attention mechanism to grasp intermittently changing points from individual's open blog streams.
Outcome:	The proposed approach can achieve over 91% accuracy with the use of suicide-oriented word embeddings and attention on a large-scale well-labelled suicide data set.

Copied to clipboard

Junbo Niu, Zheng Liu, Zhuangcheng Gu, Bin Wang, Linke Ouyang, Zhiyuan Zhao, Tao Chu, Tianyao He, Fan Wu, Qintong Zhang, Zhenjiang Jin, Guang Liang, Rui Zhang, Wenzheng Zhang, Yuan Qu, Zhifei Ren, Yuefeng Sun, Zirui Tang, Boyu Niu, Yuanhong Zheng, Dongsheng Ma, Ziyang Miao, Hejun Dong, Siyi Qian, Junyuan Zhang, Fangdong Wang, Jingzhou Chen, Xiaomeng Zhao, Liqun Wei, Wei Li, Shasha Wang, RuiLiang Xu, Yuanyuan Cao, Lu Chen, Qianqian Wu, Huaiyu Gu, Lindong Lu, Dechen Lin, null Shenguanlin, Xuanhe Zhou, Linfeng Zhang, Yuhang Zang, Xiaoyi Dong, Jiaqi Wang, Bo Zhang, Lei Bai, Pei Chu, Weijia Li, Jiang Wu, Lijun Wu, Zhenxiang Li, Guangyu Wang, Zhongying Tu, Chao Xu, Kai Chen, Bowen Zhou, Dahua Lin, Wentao Zhang, Conghui He

Green & Sustainable NLP

Challenge:	Document images are characterized by higher resolutions, denser content, and more complex structural layouts.
Approach:	They propose a 1.2B-parameter document parsing vision-language model that decouples layout analysis from local content recognition.
Outcome:	The proposed model surpasses general-purpose and domain-specific models on multiple benchmarks while maintaining significantly lower computational overhead.

MedForge: Interpretable Medical Deepfake Detection via Forgery-aware Reasoning (2026.acl-long)

Copied to clipboard

Zhihui Chen, Kai He, Qingyuan Lei, Bin Pu, Jian Zhang, Yuling Xu, Mengling Feng

ChatHLS: Towards Systematic Design Automation and Optimization for High-Level Synthesis (2026.acl-long)

Challenge:	Existing defenses against forgery are inadequate for healthcare.
Approach:	They propose a large-scale benchmark for pre-hoc, evidence-grounded medical forgery detection using a doctor inspection guideline and gold edit locations.
Outcome:	Experiments show that the proposed solution can detect and explain medical scans with high fidelity and accuracy.

Copied to clipboard

Runkai Li, Jia Xiong, Xiuyuan He, Jieru Zhao, Jiaqi Lv, Haowen Fang, Lei Qi, Xi Wang

Making Large Language Models Efficient Dense Retrievers (2026.acl-long)

Challenge:	High-Level Synthesis (HLS) is a hardware design tool that can be used to design hardware from C-like languages, but its widespread adoption is limited by strict coding constraints and design-specific optimizations.
Approach:	They propose a multi-agent HLS design framework that leverages specialized LLMs for automated debugging and directive tuning.
Outcome:	The proposed framework outperforms Gemini-3-pro in debugging and speedups across various HLS kernels and neural network accelerators.

Copied to clipboard

Yibin Lei, Shwai He, Ang Li, Andrew Yates

Hire a Linguist!: Learning Endangered Languages in LLMs with In-Context Linguistic Descriptions (2024.findings-acl)

Challenge:	Recent studies have shown that fine-tuning large language models for dense retrieval yields strong performance, but their substantial parameter counts make them computationally inefficient.
Approach:	They propose a framework for developing efficient retrievers that performs coarse-to-fine compression through a coarse-grained coarse-tuning strategy.
Outcome:	The proposed framework reduces model size and inference cost while preserving performance of full-size models.

Copied to clipboard

Kexun Zhang, Yee Choi, Zhenqiao Song, Taiqi He, William Yang Wang, Lei Li

S3Eval: A Synthetic, Scalable, Systematic Evaluation Suite for Large Language Model (2024.naacl-long)

Challenge:	Existing LLMs rarely perform well in unseen, endangered languages . Existing models such as Llama and GPT-4 lack a rich corpus of training data .
Approach:	They propose a training-free approach to enable an LLM to process unseen languages that hardly occur in its pre-training.
Outcome:	The proposed approach elevates translation capability from GPT-4’s 0 to 10.5 BLEU for 10 language directions.

Copied to clipboard

Fangyu Lei, Qian Liu, Yiming Huang, Shizhu He, Jun Zhao, Kang Liu

Align Voting Behavior with Public Statements for Legislator Representation Learning (2021.acl-long)

Challenge:	Existing benchmarks fail to evaluate extremely long-context LLMs or analyze their limitations.
Approach:	They propose a Synthetic, Scalable, Systematic evaluation suite for LLMs using SQL execution.
Outcome:	The proposed evaluation suite is able to scale text length and difficulty across scenarios and provides strong correlations with real-world benchmarks.

Copied to clipboard

Xinyi Mou, Zhongyu Wei, Lei Chen, Shangyi Ning, Yancheng He, Changjian Jiang, Xuanjing Huang

Representation Learning

Challenge:	Existing studies rely on roll call data to estimate political preference of legislators.
Approach:	They propose to integrate voting behavior and public statements on Twitter to jointly model legislators.
Outcome:	The proposed model improves on the task of roll call vote prediction . it also shows that the model captures nuances in statements .

Syntactic Graph Convolutional Network for Spoken Language Understanding (2020.coling-main)

Copied to clipboard

Keqing He, Shuyu Lei, Yushu Yang, Huixing Jiang, Zhongyuan Wang

PSSAT: A Perturbed Semantic Structure Awareness Transferring Method for Perturbation-Robust Slot Filling (2022.coling-1)

Challenge:	Existing work on slot filling and intent detection builds joint models without prior knowledge of linguistic knowledge.
Approach:	They propose a joint model that integrates syntactic structure for learning slot filling and intent detection jointly.
Outcome:	The proposed model outperforms existing models on two public benchmark datasets and further improves on slot filling and intent detection.

Copied to clipboard

Guanting Dong, Daichi Guo, Liwen Wang, Xuefeng Li, Zechen Wang, Chen Zeng, Keqing He, Jinzheng Zhao, Hao Lei, Xinyue Cui, Yi Huang, Junlan Feng, Weiran Xu

Robustness in NLP

Challenge:	Existing slot filling models memorize inherent patterns of entities and contexts from training data.
Approach:	They propose a perturbed semantic structure awareness transferring method for slot filling models . they use two MLM-based training strategies to learn contextual semantic structure and word distribution .
Outcome:	The proposed method outperforms existing methods and gains strong generalization while preventing model from memorizing inherent patterns of entities and contexts.

Attention Consistency for LLMs Explanation (2025.findings-emnlp)

Copied to clipboard

Tian Lan, Jinyuan Xu, Xue He, Jenq-Neng Hwang, Lei Li

Explainability & Interpretability in NLP

Challenge:	Existing interpretability methods face limitations such as low resolution and high computational cost.
Approach:	They propose a multi-layer attention consistency score to estimate the importance of input tokens in large language models.
Outcome:	The proposed heuristic achieves a favorable trade-off between interpretability quality and computational efficiency .

When and How to Augment Your Input: Question Routing Helps Balance the Accuracy and Efficiency of Large Language Models (2025.findings-naacl)

Copied to clipboard

Shufan Chen, He Zheng, Lei Cui

Natural Language Interfaces

Challenge:	augmented generation of knowledge-based long-tail questions can be useful for large language models, but can cause significant latency.
Approach:	They propose an adaptive question routing framework that uses a query router to augment input to the right time.
Outcome:	The proposed framework surpasses existing approaches in accuracy and efficiency on benchmarks such as AmbigNQ, HotpotQA, MMLU-STEM, and PopQA.

KLMo: Knowledge Graph Enhanced Pretrained Language Model with Fine-Grained Relationships (2021.findings-emnlp)

Copied to clipboard

Lei He, Suncong Zheng, Tao Yang, Feng Zhang

COPR: Continual Human Preference Learning via Optimal Policy Regularization (2025.findings-acl)

Challenge:	Existing knowledge-enhanced pretrained language models focus on entity information and ignore fine-grained relationships between entities.
Approach:	They propose to incorporate KG into the language learning process to obtain a KG-enhanced pretrained Language Model.
Outcome:	The proposed model improves on several knowledge-driven tasks, such as entity typing and relation classification, compared with the state-of-the-art knowledge-enhanced PLMs.

Copied to clipboard

Han Zhang, Lin Gui, Yu Lei, Yuanzhao Zhai, Yehong Zhang, Zhuo Zhang, Yulan He, Hui Wang, Yue Yu, Kam-Fai Wong, Bin Liang, Ruifeng Xu

Be Careful about Poisoned Word Embeddings: Exploring the Vulnerability of the Embedding Layers in NLP Models (2021.naacl-main)

Challenge:	Reinforcement Learning from Human Feedback (RLHF) is effective for aligning Large Language Models with human preferences, but its complex process limits its ability to continually learn human feedback.
Approach:	They propose a non-RL offline method to convert historical optimal policies into optimization constraints when continually learning new preferences.
Outcome:	The proposed method outperforms strong CL baselines in terms of reward-based evaluations and human assessment.

Copied to clipboard

Wenkai Yang, Lei Li, Zhiyuan Zhang, Xuancheng Ren, Xu Sun, Bin He

Representation Learning

Challenge:	Recent studies reveal a security threat to natural language processing models, called the Backdoor Attack.
Approach:	They propose to hack a model by modifying one single word embedding vector without sacrificing accuracy on clean samples.
Outcome:	The proposed method is more efficient and stealthier on sentiment analysis and sentence-pair classification tasks.

SimPBL: A Multi-Agent Framework for Project-Based Learning (2026.acl-long)

Copied to clipboard

Daniel Zhang-Li, Joy Jia Yin Lim, Binglin Liu, Shangqing Tu, Zijun Yao, Hao Peng, Jifan Yu, Haoxuan Li, Zhanxin Hao, Ye He, Zekun Li, Jiangyi Wang, Lei Hou, Bin Xu, Xin Cong, Zhiyuan Liu, Huiqin Liu, Yu Zhang, Juanzi Li

Challenge:	Existing LLMs provide partial assistance without modeling these roles, and overly comprehensive help can reduce learner autonomy.
Approach:	They propose a multi-agent framework with an orchestrator agent that provides adaptive scaffolding from interaction logs and collaborator agents that support project work through boundary-aware collaboration.
Outcome:	The proposed framework improves learner examination scores by 14% . it is based on a multi-agent framework with an orchestrator agent .

Comprehensive Benchmarking of Long-Form Speech Generation in Diverse Scenarios (2026.findings-acl)

Copied to clipboard

Changhao Pan, Rui Yang, Han Wang, Zhuan Zhou, Xuming He, Wenxiang Guo, Ziyue Jiang, Ruiqi Li, Yu Zhang, Chenyuhao Wen, Ke Lei, Xiang Yin, Jingyu Lu, Zhiyuan Zhu, Zhou Zhao

RAP-ID: Mechanistic Prompt Injection Detection via Impostor Behavior Analysis (2026.findings-acl)

Challenge:	Existing evaluation benchmarks for long-form speech are limited to limited domains, creating a significant gap with the diverse downstream applications.
Approach:	They propose a benchmark that decomposes "long-form speech quality" into specific, disentangled dimensions.
Outcome:	The proposed benchmark decomposes “long-form speech quality” into specific, disentangled dimensions.

Copied to clipboard

Yuchen Yang, Lei Peng, Yujie He, Yang yu, Zhongxin Wu, Yanlei Shi

CLMTracing: Black-box User-level Watermarking for Code Language Model Tracing (2025.emnlp-main)

Challenge:	Existing defenses rely on externally deployed guardrail models or response inspection . current defenses depend on external guardrails or response inspecting .
Approach:	They propose a mechanistic, train-free detection framework that operates exclusively on internal state dynamics during the initial forward pass.
Outcome:	The proposed framework achieves competitive performance with significant overall improvements compared to heuristic methods.

Copied to clipboard

Boyu Zhang, Ping He, Tianyu Du, Xuhong Zhang, Lei Yun, Kingsum Chow, Jianwei Yin

Robustness in NLP

Challenge:	Open-source code language models (code LMs) are a growing threat for intellectual property protection.
Approach:	They propose a black-box code LM watermarking framework that uses rule-based watermarks and utility-preserving injection method for user-level model tracing.
Outcome:	The proposed framework shows that it performs well across multiple state-of-the-art code LMs and is harmless compared to existing baselines.

DiffusEmp: A Diffusion Model-Based Framework with Multi-Grained Control for Empathetic Response Generation (2023.acl-long)

Copied to clipboard

Guanqun Bi, Lei Shen, Yanan Cao, Meng Chen, Yuqiang Xie, Zheng Lin, Xiaodong He

Text Generation

Challenge:	Existing methods to generate empathetic responses are monotonous and generic, resulting in shallow empathy and few connections to the context.
Approach:	They propose to use explicit control to guide the empathy expression and a framework DiffusEmp to unify the utilization of dialogue context and attribute-oriented control signals.
Outcome:	The proposed framework outperforms baselines on EmpatheticDialogue in terms of controllability, informativeness, diversity, and diversity without the loss of context-relatedness.

GeoRA: Geometry-Aware Low-Rank Adaptation for RLVR (2026.acl-long)

Copied to clipboard

Jiaying Zhang, Lei Shi, Jiguo Li, Jun Xu, Jiuchong Gao, Jinghua Hao, Renqing He

Few-Shot Table Understanding: A Benchmark Dataset and Pre-Training Baseline (2022.coling-1)

Challenge:	Existing parameter-efficient methods for RLVR face limitations . low-rank adaptation methods do not account for the distinct optimization dynamics .
Approach:	They propose a low-rank adaptation method tailored for RLVR that exploits the anisotropic structure of RL update subspace and extracts its principal directions via Singular Value Decomposition (SVD).
Outcome:	Experiments on large reasoning models show that GeoRA outperforms strong low-rank baselines across RLVR settings while showing stronger generalization and less forgetting on out-of-domain tasks.

Copied to clipboard

Ruixue Liu, Shaozu Yuan, Aijun Dai, Lei Shen, Tiangang Zhu, Meng Chen, Xiaodong He

Fine-Mem: Fine-Grained Feedback Alignment for Long-Horizon Memory Management (2026.acl-long)

Challenge:	Pre-trained language models have demonstrated their effectiveness for few-shot table understanding, but few-shoot table understanding is rarely explored due to the deficiency of public table pre-training corpus and well-defined downstream benchmark tasks.
Approach:	They establish a benchmark dataset and use it to explore few-shot table understanding in Chinese.
Outcome:	The proposed model improves the few-shot table understanding in Chinese.

Copied to clipboard

Weitao Ma, Xiaocheng Feng, Lei Huang, Xiachong Feng, Zhanyu Ma, Jun Xu, Jiuchong Gao, Jinghua Hao, Renqing He, Bing Qin

Teaching Small Language Models to Reason for Knowledge-Intensive Multi-Hop Question Answering (2024.findings-acl)

Challenge:	Existing approaches to memory management rely on final task performance as the primary reward, resulting in severe reward sparsity and ineffective credit assignment.
Approach:	They propose a framework for fine-grained feedback alignment using a Chunk-level step reward and Evidence-Anchored Reward Attribution to redistribute global rewards based on memory items utilized as evidence in reasoning.
Outcome:	The proposed framework outperforms baselines and supports generalization across different model configurations and backbones.

Copied to clipboard

Xiang Li, Shizhu He, Fangyu Lei, JunYang JunYang, Tianhuang Su, Kang Liu, Jun Zhao

MoE Adapter for Large Audio Language Models: Sparsity, Disentanglement, and Gradient-Conflict-Free (2026.findings-acl)

Challenge:	Large language models can teach small language models to solve complex reasoning tasks by Chain-of-thought Distillation (CoTD) e.g., mathematical question answering.
Approach:	They propose a method that distills two student models to solve a multi-hop question . they use chain-of-thought distillation to generate step-by-step reasoning paths .
Outcome:	The proposed method surpasses existing methods on knowledge-intensive multi-hop questions.

Copied to clipboard

Yishu Lei, Shuwei He, Hu Jing, Dan Zhang, Xianlong Luo, Danxiang Zhu, Shikun Feng, Rui Liu, Jingzhou HE, Yu Sun, Hua Wu, Haifeng Wang

ProActor: Timing-Aware Reinforcement Learning for Proactive Task Scheduling Agents (2026.acl-long)

Challenge:	Existing research on Large Language Models (LLMs) limited to textual input modality . acoustic information is intrinsically heterogeneous, entangling attributes such as speech, music, and environmental context.
Approach:	They propose a sparse Mixture-of-Experts architecture to decouple acoustic information by routing audio tokens to specialized experts.
Outcome:	The proposed architecture outperforms existing models on audio semantic and paralinguistic tasks while retaining shared experts for global context.

Copied to clipboard

Lei Ding, Bin He, Chenguang Wang, Yang Liu

OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problems (2024.acl-long)

Challenge:	Existing approaches to measuring and optimizing proactive task-oriented agents lack generalizable end-to-end solutions.
Approach:	They propose a framework for conversational task scheduling that integrates proactiveness reinforcement learning with a domain-agnostic annotation methodology.
Outcome:	The proposed framework enables scalable proactiveness reinforcement learning (RL) Experiments on two newly auto-annotated datasets demonstrate significant improvements in proactive timing while maintaining action consistency comparable to state-of-the-art baselines.

Copied to clipboard

Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, Jie Liu, Lei Qi, Zhiyuan Liu, Maosong Sun

MTR-DuplexBench: Towards a Comprehensive Evaluation of Multi-Round Conversations for Full-Duplex Speech Language Models (2026.findings-acl)

Challenge:	Large Language Models (LLMs) and Large Multimodal Models have exceeded general human capabilities in various tasks.
Approach:	They present an Olympiad-level bilingual multimodal scientific benchmark featuring 8,476 problems from Olympiad level mathematics and physics competitions.
Outcome:	The best performing model, GPT-4V, attains an average score of 17.97% on OlympiadBench, with a mere 10.74% in physics, highlighting the benchmark rigor and the intricacy of physical reasoning.

Copied to clipboard

Zhang He, Wenqian Cui, Haoning Xu, Xiao-Hui Li, Lei Zhu, Haoli Bai, Ma Shaohua, Irwin King

Ex3: Automatic Novel Writing by Extracting, Excelsior and Expanding (2024.acl-long)

Challenge:	Existing benchmarks focus on evaluating single-round interactions, neglecting other critical aspects.
Approach:	They propose a benchmark to evaluate full-duplex speech language models in multi-round settings . they segment continuous full-dual dialogues into discrete turns for evaluation .
Outcome:	The proposed benchmark compared full-duplex speech language models with full-dual speech models . the results show that the models perform better in multi-round settings than standard models compared to benchmarks .

Copied to clipboard

Huang Lei, Jiaming Guo, Guanhua He, Xishan Zhang, Rui Zhang, Shaohui Peng, Shaoli Liu, Tianshi Chen

Challenge:	Generating long-term texts using artificial intelligence has always been a challenge . however, the generated novels exhibit poor logical coherence and appeal in their plots and deficiencies in character and event depiction, ultimately compromising the overall narrative quality.
Approach:	They propose a method for extracting excelsior and expanding from novel data to generate arbitrarily long novels using large language models.
Outcome:	The proposed method produces high-quality long-form novels with a high level of logical coherence and appeal despite the use of large language models.

UCFE: A User-Centric Financial Expertise Benchmark for Large Language Models (2025.findings-naacl)

Copied to clipboard

Yuzhe Yang, Yifei Zhang, Yan Hu, Yilin Guo, Ruoli Gan, Yueru He, Mingcong Lei, Xiao Zhang, Haining Wang, Qianqian Xie, Jimin Huang, Honghai Yu, Benyou Wang

GATE: Graph-based Adaptive Tool Evolution Across Diverse Tasks (2026.acl-long)

Challenge:	Recent advances in large language models (LLMs) have expanded their potential applications in finance.
Approach:	They propose a framework to evaluate the ability of large language models to handle financial tasks using human expert evaluations and task-specific interactions.
Outcome:	The proposed framework evaluates the ability of large language models to handle complex financial tasks and combines human expert evaluations with dynamic, task-specific interactions to simulate the complexities of evolving financial scenarios.

Copied to clipboard

Jianwen Luo, Yiming Huang, Jinxiang Meng, Fangyu Lei, Shizhu He, Xiao Liu, Shanshan Jiang, Bin Dong, Jun Zhao, Kang Liu

Bringing Structure into Summaries: a Faceted Summarization Dataset for Long Scientific Documents (2021.acl-short)

Challenge:	Existing toolsets that use large language models are limited to single-task settings.
Approach:	They propose a framework that dynamically constructs and evolves a hierarchical graph of reusable tools across multiple scenarios.
Outcome:	The proposed framework achieves up to 4.3 faster milestone completion in Minecraft compared to the previous state-of-the-art method and provides an average improvement of 9.23% over existing tool-making methods in code generation tasks and 10.03% in agent tasks.

Copied to clipboard

Rui Meng, Khushboo Thaker, Lei Zhang, Yue Dong, Xingdi Yuan, Tong Wang, Daqing He

Text Generation

Challenge:	Faceted summarization provides briefings of a document from different perspectives.
Approach:	They propose a faceted summarization benchmark built on Emerald journal articles . they propose faceted models that bring structure into faceted documents .
Outcome:	The proposed benchmark is based on Emerald journal articles and covers a diverse range of domains.

SoftDedup: an Efficient Data Reweighting Method for Speeding Up Language Model Pre-training (2024.acl-long)

Copied to clipboard

Nan He, Weichen Xiong, Hanwen Liu, Yi Liao, Lei Ding, Kai Zhang, Guohua Tang, Xiao Han, Yang Wei

QDTSynth: Quality-Driven Formal Theorem Synthesis for Enhancing Proving Performance of LLMs (2025.acl-long)

Challenge:	Current methods focus on detecting and removing duplicates, which risks the loss of valuable information and neglects the varying degrees of duplication.
Approach:	They propose a method that maintains dataset integrity while selectively reducing the sampling weight of data with high commonness.
Outcome:	The proposed method significantly improves training efficiency on deduplicated datasets and improves downstream accuracy by 1.77%.

Copied to clipboard

Lei Wang, Ruobing Zuo, Gaolei He, Jianlin Wang, Zhengfeng Yang

A Survey of Reinforcement Learning for Large Language Models under Data Scarcity: Challenges and Solutions (2026.acl-long)

Challenge:	Existing formal languages such as Lean, Coq and Metamath are proving to be useful in formal theorem proving . however, there is a scarcity of high-quality supervised fine-tuning data for formal proofs .
Approach:	They propose a Quality-Driven Theorem S**ynthesis method in Lean4 . they propose diversity screening and the self-assessment method to select theoremas that exhibit diversity and high quality from the initial synthetic statements.
Outcome:	The proposed method significantly improves performance of open-source LLMs in theorem proving tasks.

Copied to clipboard

Zhiyin Yu, Yuchen Mou, Juncheng Yan, Junyu Luo, Chunchun Chen, Xing Wei, Yunhui Liu, Hongru Sun, Yuxing Zhang, Jun Xu, Yatao Bian, Ming Zhang, Wei Ye, Tieke He, Jie Yang, Guanjie Zheng, Zhonghai Wu, Bo Zhang, Lei Bai, Xiao Luo

S3HQA: A Three-Stage Approach for Multi-hop Text-Table Hybrid Question Answering (2023.acl-short)

Challenge:	Existing research on reinforcement learning for LLMs under data scarcity has not been unified.
Approach:	They propose a top-up hierarchical framework built around three complementary perspectives: data-centric, training-centric and framework-centric.
Outcome:	The proposed framework provides a clear conceptual foundation for understanding the design space of data-efficient RL for large language models and to guide researchers working in this emerging area.

Copied to clipboard

Fangyu Lei, Xiang Li, Yifan Wei, Shizhu He, Yiming Huang, Jun Zhao, Kang Liu

Uncertainty Estimation on Sequential Labeling via Uncertainty Transmission (2024.findings-naacl)

Challenge:	Existing question answering systems use a retriever-reader framework to answer multi-hop questions . existing models lack retrieval, selector, and reasoner capabilities .
Approach:	They propose a three-stage text tableQA framework which comprises of retriever, selector, and reasoner.
Outcome:	The proposed framework outperforms baseline methods in the few-shot setting and ranks first on the HybridQA leaderboard.

Copied to clipboard

Jianfeng He, Linlin Yu, Shuo Lei, Chang-Tien Lu, Feng Chen

Information Extraction & Text Mining

Challenge:	Named entity recognition tasks are often suboptimal for NER . previous work focused on UE-NER, which estimates uncertainty scores for ner .
Approach:	They propose to use a Sequential Labeling Posterior Network to estimate uncertainty for NER . they propose to consider wrong-span cases and to evaluate the specificity of wrong-pan cases.
Outcome:	The proposed system improves on three datasets and AUPR on MIT-Restaurant datasets.

LongAlign: A Recipe for Long Context Alignment of Large Language Models (2024.findings-emnlp)

Copied to clipboard

Yushi Bai, Xin Lv, Jiajie Zhang, Yuze He, Ji Qi, Lei Hou, Jie Tang, Yuxiao Dong, Juanzi Li

A Finer-grain Universal Dialogue Semantic Structures based Model For Abstractive Dialogue Summarization (2021.findings-emnlp)

Challenge:	Existing studies to build long context language models focus on context extension and continual training on long text.
Approach:	They propose a recipe for instruction fine-tuning on input sequences of similar length . they adopt packing and sorted batching strategies to speed up supervised fine-uning .
Outcome:	The proposed model outperforms existing recipes for LLMs in long context tasks by 30% while maintaining proficiency in handling short, generic tasks.

Copied to clipboard

Yuejie Lei, Fujia Zheng, Yuanmeng Yan, Keqing He, Weiran Xu

Mis-prompt: Benchmarking Large Language Models for Proactive Error Handling (2025.acl-long)

Challenge:	Abstractive summarization models have achieved impressive results on document summarizing tasks, but their performance on dialogue modeling is poor due to the crude and straight methods for dialogue encoding.
Approach:	They propose a model that leverages Finer-grain universal Dialogue semantic Structures to model dialogue and generate better summaries.
Outcome:	The proposed model outperforms various dialogue summarization approaches and achieves state-of-the-art (SOTA) ROUGE results on a SAMsum dataset.

Copied to clipboard

Jiayi Zeng, Yizhe Feng, Mengliang He, Wenhui Lei, Wei Zhang, Zeming Liu, Xiaoming Shi, Aimin Zhou

DA-Code: Agent Data Science Code Generation Benchmark for Large Language Models (2024.emnlp-main)

Challenge:	Current error-handling works are performed in a passive manner, with explicit error- handling instructions.
Approach:	They propose a new benchmark to analyze LLMs' performance on a mis-prompt benchmark and a dataset to promote further research.
Outcome:	The proposed benchmark shows that current LLMs show poor performance on proactive error handling, and that SFT improves on error handling instances.

Copied to clipboard

Yiming Huang, Jianwen Luo, Yan Yu, Yitong Zhang, Fangyu Lei, Yifan Wei, Shizhu He, Lifu Huang, Xiao Liu, Jun Zhao, Kang Liu

Programming Languages in NLP

Challenge:	DA-Code is a code generation benchmark designed to assess LLMs on agent-based data science tasks.
Approach:	They propose a code generation benchmark specifically designed for LLMs on agent-based data science tasks.
Outcome:	The benchmark performs better than existing frameworks, but lacks accuracy . it is based on real-world data, and includes examples that cover a wide range of tasks .

KQA Pro: A Dataset with Explicit Compositional Programs for Complex Question Answering over Knowledge Base (2022.acl-long)

Copied to clipboard

Shulin Cao, Jiaxin Shi, Liangming Pan, Lunyiu Nie, Yutong Xiang, Lei Hou, Juanzi Li, Bin He, Hanwang Zhang

Natural Language Interfaces

Challenge:	Existing benchmarks for Complex KBQA lack compositional reasoning capabilities . Existing methods for Complex questions are poor in diversity or scale .
Approach:	They propose a compositional programming language to represent the reasoning process of complex questions.
Outcome:	The proposed dataset includes around 120K diverse natural language questions . it provides a compositional and interpretable programming language to represent the reasoning process of complex questions based on the proposed model .

KEPL: Knowledge Enhanced Prompt Learning for Chinese Hypernym-Hyponym Extraction (2023.emnlp-main)

Copied to clipboard

Ningchen Ma, Dong Wang, Hongyun Bao, Lei He, Suncong Zheng

Global and Local Hierarchical Prompt Tuning Framework for Multi-level Implicit Discourse Relation Recognition (2024.lrec-main)

Challenge:	Existing work on hypernym-hyponym (“is-a”) relations is mostly in the English language.
Approach:	They propose a Knowledge Enhanced Prompt Learning method for Chinese hypernym-hyponym relation extraction using Hearst-like patterns as the prior knowledge.
Outcome:	The proposed method is able to extract hypernym-hyponym relations from Chinese unstructured texts using Hearst-like patterns and embed patterns and text simultaneously.

Copied to clipboard

Lei Zeng, Ruifang He, Haowen Sun, Jing Xu, Chang Liu, Bo Wang

BenchMAX: A Comprehensive Multilingual Evaluation Suite for Large Language Models (2025.findings-emnlp)

Challenge:	Recent methods to recognize hierarchical discourse relations without explicit connectives are inefficient and ignore the utilization of the output probability distribution information of the verbalizer.
Approach:	They propose a global and local hierarchical prompt tuning framework which leverages top-up propagated probability as the global hierarchy to inject it into multi-level verbalizer.
Outcome:	The proposed framework achieves competitive results on two benchmacks.

Copied to clipboard

Xu Huang, Wenhao Zhu, Hanxu Hu, Conghui He, Lei Li, Shujian Huang, Fei Yuan

Refining and Synthesis: A Simple yet Effective Data Augmentation Framework for Cross-Domain Aspect-based Sentiment Analysis (2024.findings-acl)

Challenge:	Existing multilingual benchmarks focus primarily on language understanding tasks.
Approach:	They develop a multi-way multilingual benchmark that measures critical capabilities of large language models across languages.
Outcome:	Extensive experiments on BenchMAX reveal uneven utilization of core capabilities across languages, emphasizing the performance gaps that scaling model size alone does not resolve.

Copied to clipboard

Haining Wang, Kang He, Bobo Li, Lei Chen, Fei Li, Xu Han, Chong Teng, Donghong Ji

Sentiment Analysis

Challenge:	Aspect-based Sentiment Analysis (ABSA) data augmentation has attracted increasing attention in recent years due to data sparsity.
Approach:	They propose a framework to augment ABSA data using pseudo labels for target domain . they refine generated labeled data using a natural language inference filter .
Outcome:	The proposed framework outperforms 7 strong baselines on 4 kinds of ABSA tasks.

Scaling Behaviors of LLM Reinforcement Learning Post-Training: An Empirical Study in Mathematical Reasoning (2026.acl-long)

Copied to clipboard

Zelin Tan, Hejia Geng, Xiaohang Yu, Mulei Zhang, Guancheng Wan, Yifan Zhou, Qiang He, Xiangyuan Xue, Heng Zhou, Yutao Fan, Zhong-Zhi Li, Zaibin Zhang, Guibin Zhang, Chen Zhang, Zhenfei Yin, Philip Torr, Lei Bai

Towards More Accurate Uncertainty Estimation In Text Classification (2020.emnlp-main)

Challenge:	elucidating scaling laws for large language models (LLMs) during pre-training remains unexplored.
Approach:	They characterize how model scale, data, and compute interact during pre-training . they find that large models consistently demonstrate superior compute and data efficiency .
Outcome:	The proposed scaling laws offer practical guidance for scaling reasoning capabilities through reinforcement learning post-training.

Copied to clipboard

Jianfeng He, Xuchao Zhang, Shuo Lei, Zhiqian Chen, Fanglan Chen, Abdulaziz Alhamadani, Bei Xiao, ChangTien Lu

Information Extraction & Text Mining

Challenge:	Existing models of uncertainty score depend on winning score, which is the maximum probability in a semantic vector.
Approach:	They propose to generate accurate uncertainty score by improving the confidence of winning scores by reducing the effect of overconfidence of winning score and considering the impact of different categories simultaneously.
Outcome:	The proposed model reduces the effect of overconfidence of winning score and considers impact of different categories of uncertainty simultaneously.

MCAD: Multi-teacher Cross-modal Alignment Distillation for efficient image-text retrieval (2024.findings-naacl)

Copied to clipboard

Youbo Lei, Feifei He, Chen Chen, Yingbin Mo, Sijia Li, Defeng Xie, Haonan Lu

Green & Sustainable NLP

Challenge:	Large-scale visual-language pretraining models have shown remarkable capabilities in understanding both vision and language.
Approach:	They propose a multi-teacher cross-modality alignment distillation technique to integrate the advantages of single-stream and dual-stream models.
Outcome:	The proposed model is lightweight and has only 100M running memory and 8.0ms search latency.

Beyond Correctness: Confidence-Aware Reward Modeling for Enhancing Large Language Model Reasoning (2025.emnlp-main)

Copied to clipboard

Qianxi He, Qingyu Ren, Shanzhe Lei, Xuhong Wang, Yingchun Wang

A.S.E: A Repository-Level Benchmark for Evaluating Security in AI-Generated Code (2026.findings-acl)

Challenge:	Recent advances in large language models have shifted the post-training paradigm from instruction tuning and human preference alignment to reinforcement learning (RL) based on rule-based evaluations of answer correctness, these models often receive rewards for speculative answers without generating coherent reasoning chains.
Approach:	They propose a confidence-based reward model tailored for enhancing STEM reasoning capabilities.
Outcome:	The proposed model outperforms state-of-the-art open-source reward models across diverse STEM benchmarks.

Copied to clipboard

Keke Lian, Wang Bin, Lei Zhang, Libo Chen, Junjie Wang, Ziming Zhao, Yujiu Yang, Miaoqian Lin, Haotong Duan, Haoran Zhao, Shuang Liao, Mingda Guo, Quan Jiazheng, Yilu Zhong, Chenhao He, Chen Zichuan, Jie Wu, Haoling Li, Zhaoxuan Li, Jiongchi Yu, Hui LI, Dong Zhang

Programming Languages in NLP

Challenge:	Existing security evaluation benchmarks lack relevance to real-world AI programming tasks . current LLMs struggle with secure coding, research shows .
Approach:	They propose a repository-level evaluation benchmark to assess security of AI-generated code.
Outcome:	The proposed framework mirrors real-world AI programming tasks and offers valuable insights into the state of AI code generation.

Generative Zero-Shot Prompt Learning for Cross-Domain Slot Filling with Inverse Prompting (2023.findings-acl)

Copied to clipboard

Xuefeng Li, Liwen Wang, Guanting Dong, Keqing He, Jinzheng Zhao, Hao Lei, Jiachi Liu, Weiran Xu

A Novel Negative Sample Generation Method for Contrastive Learning in Hierarchical Text Classification (2025.coling-main)

Challenge:	Existing approaches to slot filling only learn surface mapping of slot types between D S and D T and get poor generalization capability or robustness.
Approach:	They propose a generative zero-shot prompt learning framework for cross-domain slot filling which improves generalization and robustness than previous work.
Outcome:	The proposed framework improves generalization and robustness on unseen slots and an efficient prompt tuning strategy boosts performance.

Copied to clipboard

Juncheng Zhou, Lijuan Zhang, Yachen He, Rongli Fan, Lei Zhang, Jian Wan