Papers by Lei Peng

MAVEN-ARG: Completing the Puzzle of All-in-One Event Understanding Dataset with Event Argument Annotation (2024.acl-long)

Challenge:	Event understanding is fundamental for humans to understand the world.
Approach:	They propose an event understanding toolkit called OmniEvent that is comprehensive and fair . it supports mainstream modeling paradigms and the processing of 15 widely-used datasets .
Outcome:	The toolkit supports mainstream modeling paradigms and the processing of 15 widely-used English and Chinese datasets.

Copied to clipboard

Xiaozhi Wang, Hao Peng, Yong Guan, Kaisheng Zeng, Jianhui Chen, Lei Hou, Xu Han, Yankai Lin, Zhiyuan Liu, Ruobing Xie, Jie Zhou, Juanzi Li

COPEN: Probing Conceptual Knowledge in Pre-trained Language Models (2022.emnlp-main)

Challenge:	Existing datasets for event understanding have limited coverage due to complexity of tasks.
Approach:	They propose a dataset that augments MAVEN datasets with event argument annotations . they propose 98,591 events and 290,613 arguments obtained with laborious human annotation .
Outcome:	The proposed dataset is the first all-in-one dataset supporting event detection, event argument extraction, and event relation extraction.

Copied to clipboard

Hao Peng, Xiaozhi Wang, Shengding Hu, Hailong Jin, Lei Hou, Juanzi Li, Zhiyuan Liu, Qun Liu

Data Mixing Agent: Learning to Re-weight Domains for Continual Pre-training (2026.acl-long)

Challenge:	Existing knowledge probing studies focus on evaluating factual knowledge of pre-trained language models (PLMs) but ignore conceptual knowledge.
Approach:	They evaluate conceptual knowledge of pre-trained language models by annotating 24k data instances covering 393 concepts.
Outcome:	The proposed tasks evaluate pre-trained language models' conceptual knowledge of entities, learn conceptual properties, and conceptualize entities in contexts.

Copied to clipboard

Kailai Yang, Xiao Liu, Lei Ji, Hao Li, Xiao Liang, Zhiwei Liu, Yeyun Gong, Peng Cheng, Mao Yang

Writing-RL: Advancing Long-form Writing via Adaptive Curriculum Reinforcement Learning (2026.acl-long)

Challenge:	Existing methods for reweighting data mixtures rely on manual designation with certain heuristics based on intuition or empirical results.
Approach:	They propose a model-based framework that learns to re-weight domains by reinforcement learning on large quantities of data mixing trajectories with corresponding feedback from an evaluation environment.
Outcome:	The proposed framework outperforms baselines in achieving balanced performance across source and target fields and domain spaces without retraining.

Copied to clipboard

Xuanyu Lei, Chenliang Li, Yuning Wu, Kaiming Liu, Weizhou Shen, Peng Li, Ming Yan, Fei Huang, Ya-Qin Zhang, Yang Liu

Challenge:	Recent advances in Large Language Models (LLMs) have enabled strong performance in long-form writing, but current training paradigms remain limited.
Approach:	They propose an Adaptive Curriculum Reinforcement Learning framework to advance long-form writing capabilities beyond SFT.
Outcome:	Experiments on 7B-scale writer models show that Writing-RL improves long-form writing performance over strong SFT baselines.

WildReward: Learning Reward Models from In-the-Wild Human Interactions (2026.acl-long)

Copied to clipboard

Hao Peng, Yunjia Qi, Xiaozhi Wang, Zijun Yao, Lei Hou, Juanzi Li

Challenge:	Prior work focused on collecting preference pairs, requiring substantial annotation efforts.
Approach:	They propose a pipeline to extract reliable human feedback from in-the-wild interactions . they propose to use WildChat as an interaction source to train the model .
Outcome:	The proposed model achieves comparable or even superior performance compared to conventional models with improved calibration and cross-sample consistency.

Adaptive Meta-learner via Gradient Similarity for Few-shot Text Classification (2022.coling-1)

Copied to clipboard

Tianyi Lei, Honghui Hu, Qiaoyang Luo, Dezhong Peng, Xu Wang

Emotion Transfer with Enhanced Prototype for Unseen Emotion Recognition in Conversation (2025.emnlp-main)

Challenge:	Existing methods for few-shot text classification suffer from overfitting due to the lack of matching between the few amount of samples and complicated models.
Approach:	They propose a method to improve model generalization ability to a new task by leveraging a meta-learner via gradient similarity method.
Outcome:	The proposed method improves few-shot text classification performance on several benchmarks.

Copied to clipboard

Kun Peng, Cong Cao, Hao Peng, Guanlin Wu, Zhifeng Hao, Lei Jiang, Yanbing Liu, Philip S. Yu

Sentiment Analysis

Challenge:	Existing research on emotion recognition in conversation does not reach a consensus on classification theories . despite this, there is no clear consensus on how to recognize previously unseen emotions in real-world applications.
Approach:	They propose a prototype-based emotion transfer framework that can be used in real-world applications.
Outcome:	The proposed framework shows promise but still faces key challenges in the field of emotion recognition in conversation.

Parameter Importance is Not Static: Evolving Parameter Isolation for Supervised Fine-Tuning (2026.acl-long)

Copied to clipboard

Zekai Lin, Chao Xue, Di Liang, Xingsheng Han, Peiyang Liu, Xianjie Wu, Lei Jiang, Yu Lu, Bob Simons, Shuang Liang, Minlong Peng

ArchiDocGen: Multi-Agent Framework for Expository Document Generation in the Architectural Industry (2025.acl-industry)

Challenge:	Recent approaches to fine-tuning of large language models suffer from task interference and catastrophic forgetting.
Approach:	They propose a fine-tuning framework that adapts isolation decisions based on online estimates of parameter importance.
Outcome:	The proposed framework reduces interference and forgetting while releasing outdated parameters to recover plasticity.

Copied to clipboard

Junjie Jiang, Haodong Wu, Yongqi Zhang, Songyue Guo, Bingcen Liu, Caleb Chen Cao, Ruizhe Shao, Chao Guan, Peng Xu, Lei Chen

Text Generation

Challenge:	drafting method statements is labor-intensive and time-consuming . traditional methods involve using static templates filled in manually by engineers .
Approach:	They propose a framework that automates method statement generation by using multi-agent collaboration.
Outcome:	The proposed framework achieves 4.38 ContentScore, excelling in specialization, completeness, organization, and clarity.

Why Supervised Fine-Tuning Fails to Learn: A Systematic Study of Incomplete Learning in Large Language Models (2026.acl-long)

Copied to clipboard

Chao Xue, Yao Wang, Mengqiao Liu, Di Liang, Xingsheng Han, Peiyang Liu, Xianjie Wu, Chenyao Lu, Lei Jiang, Yu Lu, Haibo Shi, Shuang Liang, Minlong Peng, Flora D. Salim

LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitasks (2025.acl-long)

Challenge:	Incomplete learning is widespread and heterogeneous in large language models . authors identify five recurrent sources of incomplete learning: missing prerequisite knowledge, conflicts between SFT supervision and pre-training knowledge, internal inconsistencies within SFT data, left-side forgetting during sequential fine-tuning, and insufficient optimization for rare or complex patterns.
Approach:	They propose a diagnostic-first framework that maps incomplete learning to causes . they identify five recurrent sources of incomplete learning: missing prerequisite knowledge, conflicts between supervision and pre-training knowledge, internal inconsistencies, left-side forgetting during sequential fine-tuning, and insufficient optimization for rare or complex patterns.
Outcome:	The proposed framework maps incomplete learning to causes using observable training and inference signals.

Copied to clipboard

Yushi Bai, Shangqing Tu, Jiajie Zhang, Hao Peng, Xiaozhi Wang, Xin Lv, Shulin Cao, Jiazheng Xu, Lei Hou, Yuxiao Dong, Jie Tang, Juanzi Li

Reasoning

Challenge:	Existing benchmarks on longcontext large language models fail to reflect their deep understanding capabilities across diverse tasks.
Approach:	They propose a benchmark to assess the ability of long-context large language models to handle long-text problems.
Outcome:	The proposed model achieves 50.1% accuracy when directly answering the questions . human experts achieve only 53.7% accuracy under a 15-minute time constraint .

QaDialMoE: Question-answering Dialogue based Fact Verification with Mixture of Experts (2022.findings-emnlp)

Copied to clipboard

Longzheng Wang, Peng Zhang, Xiaoyu Lu, Lei Zhang, Chaoyang Yan, Chuang Zhang

Natural Language Interfaces

Challenge:	Existing research on fact verification focuses on news, tables and Wikipedia passages.
Approach:	They propose a question-answering dialogue based fact verification with mixture of experts that exploits questions and evidence effectively in the verification process.
Outcome:	The proposed approach outperforms previous approaches on three benchmark datasets and achieves state-of-the-art results.

Unifying Discrete and Continuous Representations for Unsupervised Paraphrase Generation (2023.emnlp-main)

Copied to clipboard

Mingfeng Xue, Dayiheng Liu, Wenqiang Lei, Jie Fu, Jian Lan, Mei Li, Baosong Yang, Jun Xie, Yidan Zhang, Dezhong Peng, Jiancheng Lv

Text Generation

Challenge:	Existing unsupervised paraphrase generation methods require large-scale, manually annotated paraphrase datasets, which are labor-intensive to build.
Approach:	They propose a self-supervised pseudo-data construction method that generates diverse pseudo-paraphrases in distinct surface structures for a given sentence.
Outcome:	The proposed method generates diverse pseudo-paraphrases in distinct surface structures for a given sentence.

Scaffolding Coordinates to Promote Vision-Language Coordination in Large Multi-Modal Models (2025.coling-main)

Copied to clipboard

Xuanyu Lei, Zonghan Yang, Xinrui Chen, Peng Li, Yang Liu

Multimodality

Challenge:	Existing prompting techniques for Large Multi-Modal Models (LMMs) focus on improving textual reasoning or leveraging tools for image preprocessing, lacking a simple and general visual prompting scheme to promote vision-language coordination.
Approach:	They propose a prompting scheme that scaffolds coordinates to promote vision-language coordination in Large Multi-Modal Models (LMMs) they overlay a dot matrix within the image as visual information anchors and leverage multi-dimensional coordinates as textual positional references.
Outcome:	Experiments on a wide range of vision-language tasks show the superiority of SCAFFOLD prompting over the textual Chain-of-Thought prompting.

Praetor: A Fine-Grained Generative LLM Evaluator with Instance-Level Customizable Evaluation Criteria (2025.acl-long)

Copied to clipboard

Yongqi Leng, Renren Jin, Yue Chen, Zhuowen Han, Ling Shi, Jianxiang Peng, Lei Yang, Juesi Xiao, Deyi Xiong

Challenge:	Existing evaluation methods are inadequate to evaluate large language models (LLMs).
Approach:	They propose a fine-grained generative LLM evaluator with instance-level customazable evaluation criteria that can be used to evaluate large language models.
Outcome:	The proposed model outperforms existing LLM evaluators and instruction-tuned LLMs on multiple benchmarks and sets new SOTA results.

From Observation to Understanding: Front-Door Adjustments with Uncertainty Calibration for Enhancing Egocentric Reasoning in LVLMs (2025.findings-acl)

Copied to clipboard

Shenshen Li, Wenxin Meng, Lei Wang, Hao Yang, Chong Peng, Peng Yan, Fumin Shen, Jingkuan Song, Heng Tao Shen, Xing Xu

Low-Resource NLP

Challenge:	Existing methods that adapt LVLMs to egocentric tasks overlook critical agent-environment interactions, limiting their ability to perform egoic reasoning.
Approach:	They propose a zero-shot paradigm to enhance egocentric reasoning by simulating human causal reasoning by formalizing ego-centric reasoning using a structural causal model.
Outcome:	The proposed method improves egocentric reasoning abilities on six tasks.

Reason Only When Needed: Efficient Generative Reward Modeling via Model-Internal Uncertainty (2026.findings-acl)

Copied to clipboard

Chao Xue, Yao Wang, Mengqiao Liu, Di Liang, Xingsheng Han, Peiyang Liu, Xianjie Wu, Chenyao Lu, Lei Jiang, Yu Lu, Haibo Shi, Shuang Liang, Minlong Peng, Flora D. Salim

CLEVE: Contrastive Pre-training for Event Extraction (2021.acl-long)

Challenge:	Existing approaches to generating reward models rely on voting-based mechanisms to evaluate CoT outputs.
Approach:	They propose an efficient generative reward modeling framework grounded in model-internal uncertainty.
Outcome:	The proposed framework reduces inference cost while improving answer accuracy.

Copied to clipboard

Ziqi Wang, Xiaozhi Wang, Xu Han, Yankai Lin, Lei Hou, Zhiyuan Liu, Peng Li, Juanzi Li, Jie Zhou

From Curated Data to Scalable Models: Continual Pre-training of Dense and MoE Large Language Models for Tibetan (2026.acl-long)

Challenge:	Existing EE methods do not model event characteristics from large unsupervised data.
Approach:	They propose a contrastive pre-training framework for event extraction to better learn event knowledge from large unsupervised data and their semantic structures.
Outcome:	The proposed framework improves on ACE 2005 and MAVEN datasets on event extraction tasks.

Copied to clipboard

Lei Yang, Leiyu Pan, Bojian Xiong, Renren Jin, Shaowei Zhang, Yue Chen, Ling Shi, Jiang Zhou, Junru Wu, Zhen Wang, Jianxiang Peng, Juesi Xiao, Tianyu Dong, Zhuowen Han, Zhuo Chen, Yuqi Ren, Deyi Xiong

VerIF: Verification Engineering for Reinforcement Learning in Instruction Following (2025.emnlp-main)

Challenge:	Large language models have achieved remarkable success across a wide range of tasks, yet their performance remains heavily biased toward high-resource languages.
Approach:	They propose a pipeline for advancing Tibetan language modeling through multilingual continual pre-training with Tibetan, Chinese, and English.
Outcome:	The proposed model outperforms open-source and Tibetan-focused models on diverse tasks.

Copied to clipboard

Hao Peng, Yunjia Qi, Xiaozhi Wang, Bin Xu, Lei Hou, Juanzi Li

Challenge:	Best practices for RL in instruction following remain underexplored.
Approach:	They propose a verification method that combines rule-based code verification with LLM-based verification from a large reasoning model.
Outcome:	The proposed method achieves state-of-the-art performance among models of comparable size and generalizes well to unseen constraints.

When "Correct" Is Not Safe: Can We Trust Functionally Correct Patches Generated by Code Agents? (2026.acl-long)

Copied to clipboard

Yibo Peng, James Song, Lei Li, Xinyu Yang, Mihai Christodorescu, Ravi Mangal, Corina S. Pasareanu, Haizhong Zheng, Beidi Chen

Robustness in NLP

Challenge:	Code agents are increasingly trusted to autonomously fix bugs on platforms such as GitHub, yet their security evaluation focuses on functional correctness.
Approach:	They propose to attack functionally correct yet vulnerable (FCV) patches by combining multi-turn reasoning with tool invocation and environment interaction.
Outcome:	The proposed FCV-Attack achieves an attack success rate of 40.7% on GPT-5 Mini + OpenHands.

SimPBL: A Multi-Agent Framework for Project-Based Learning (2026.acl-long)

Copied to clipboard

Daniel Zhang-Li, Joy Jia Yin Lim, Binglin Liu, Shangqing Tu, Zijun Yao, Hao Peng, Jifan Yu, Haoxuan Li, Zhanxin Hao, Ye He, Zekun Li, Jiangyi Wang, Lei Hou, Bin Xu, Xin Cong, Zhiyuan Liu, Huiqin Liu, Yu Zhang, Juanzi Li

Challenge:	Existing LLMs provide partial assistance without modeling these roles, and overly comprehensive help can reduce learner autonomy.
Approach:	They propose a multi-agent framework with an orchestrator agent that provides adaptive scaffolding from interaction logs and collaborator agents that support project work through boundary-aware collaboration.
Outcome:	The proposed framework improves learner examination scores by 14% . it is based on a multi-agent framework with an orchestrator agent .

Prompt-fused Framework for Inductive Logical Query Answering (2024.lrec-main)

Copied to clipboard

Zezhong Xu, Wen Zhang, Peng Ye, Lei Liang, Huajun Chen

RAP-ID: Mechanistic Prompt Injection Detection via Impostor Behavior Analysis (2026.findings-acl)

Challenge:	Existing methods for addressing logical queries on knowledge graphs neglect missing edges in KGs . Existing approaches focus on addressing missing edges, thereby neglecting the emergence of new entities .
Approach:	They propose a query-aware prompt-fused framework that addresses embedding of emerging entities . they propose to use a symbolic query to gather information relevant to the query .
Outcome:	The proposed framework addresses embedding of emerging entities through contextual information aggregation.

Copied to clipboard

Yuchen Yang, Lei Peng, Yujie He, Yang yu, Zhongxin Wu, Yanlei Shi

LLMRouterBench: A Massive Benchmark and Unified Framework for LLM Routing (2026.findings-acl)

Challenge:	Existing defenses rely on externally deployed guardrail models or response inspection . current defenses depend on external guardrails or response inspecting .
Approach:	They propose a mechanistic, train-free detection framework that operates exclusively on internal state dynamics during the initial forward pass.
Outcome:	The proposed framework achieves competitive performance with significant overall improvements compared to heuristic methods.

Copied to clipboard

Hao Li, Yiqun Zhang, Zhaoyan Guo, Chenxu Wang, Shengji Tang, Qiaosheng Zhang, Yang Chen, Biqing Qi, Peng Ye, Lei Bai, Zhen Wang, Shuyue Hu

Challenge:	Large language model (LLM) routing assigns each query to the best suitable model from an ensemble.
Approach:	They introduce a large-scale benchmark and unified framework for LLM routing . they find that many routing methods exhibit similar performance under unified evaluation .
Outcome:	The proposed benchmark provides comprehensive metrics for both performance-oriented and performance-cost trade-off routing.

MAVEN-ERE: A Unified Large-scale Dataset for Event Coreference, Temporal, Causal, and Subevent Relation Extraction (2022.emnlp-main)

Copied to clipboard

Xiaozhi Wang, Yulin Chen, Ning Ding, Hao Peng, Zimu Wang, Yankai Lin, Xu Han, Lei Hou, Juanzi Li, Zhiyuan Liu, Peng Li, Jie Zhou

Nature-Inspired Population-Based Evolution of Large Language Models (2026.acl-long)

Challenge:	Existing datasets only cover limited relation types at once, which prevents models from taking full advantage of relation interactions.
Approach:	They construct a large-scale human-annotated ERE dataset with improved annotation schemes to address these drawbacks.
Outcome:	The proposed dataset is larger than existing datasets of all the ERE tasks by at least an order of magnitude.

Copied to clipboard

Yiqun Zhang, Peng Ye, Xiaocui Yang, Shi Feng, Shufei Zhang, Lei Bai, Wanli Ouyang, Shuyue Hu

ADELIE: Aligning Large Language Models on Information Extraction (2024.emnlp-main)

Challenge:	a new framework for population-based evolution of large language models is emerging . a population-driven evolution of LLMs is a key component of evolution, authors say .
Approach:	They propose a framework that allows for population-based evolution of large language models . they start with a population of parent LLMs and allow this population to evolve .
Outcome:	The proposed framework outperforms existing methods on 12 datasets.

Copied to clipboard

Yunjia Qi, Hao Peng, Xiaozhi Wang, Bin Xu, Lei Hou, Juanzi Li

On Transferability of Prompt Tuning for Natural Language Processing (2022.naacl-main)

Challenge:	Large language models (LLMs) struggle to follow complex instructions of IE tasks due to not being aligned with humans.
Approach:	They propose an aligned large language moDEL that effectively solves various IE tasks including closed IE, open IE and on-demand IE.
Outcome:	The proposed model achieves state-of-the-art (SoTA) performance among open-source models.

Copied to clipboard

Yusheng Su, Xiaozhi Wang, Yujia Qin, Chi-Min Chan, Yankai Lin, Huadong Wang, Kaiyue Wen, Zhiyuan Liu, Peng Li, Juanzi Li, Lei Hou, Maosong Sun, Jie Zhou

PACE: Predictive Adaptive Context Extraction for Long-Horizon LLM Agents (2026.acl-long)

Challenge:	Pre-trained language models (PLMs) can achieve comparable performance to full-parameter fine-tuning by tuning a few soft prompts, but require much more training time than fine-timing.
Approach:	They empirically investigate the transferability of soft prompts across different downstream tasks and PLMs to determine what decides prompt transferability.
Outcome:	The proposed method can achieve comparable performance to full-parameter fine-tuning by tuning a few soft prompts, but requires much more training time than fine-timing.

Copied to clipboard

Lei Wei, Xiao Peng, null Tt, Guannan Zhang, Chenhao Jiang, Hongyu Li, Lanbo Lin, Yuanwu Xu, Jiayao Liu, Kesu Wang, Bin Wang

The Devil is in the Details: On the Pitfalls of Event Extraction Evaluation (2023.findings-acl)

Challenge:	Large Language Model (LLM) agents struggle with ultra-long-horizon tasks requiring hundreds or thousands of interaction steps.
Approach:	They propose a framework that reconceptualizes context management as a Next Step Prediction problem.
Outcome:	The proposed framework improves task success rates and robust cross-lingual performance.

Copied to clipboard

Hao Peng, Xiaozhi Wang, Feng Yao, Kaisheng Zeng, Lei Hou, Juanzi Li, Zhiyuan Liu, Weixing Shen

Ex3: Automatic Novel Writing by Extracting, Excelsior and Expanding (2024.acl-long)

Challenge:	Event extraction (EE) is a fundamental information extraction task aimed at extracting events from plain texts.
Approach:	They propose to specify data preprocessing, standardize outputs, and provide pipeline evaluation results to avoid these pitfalls.
Outcome:	The results show that the evaluations are reliable and lack pipeline evaluations.

Copied to clipboard

Huang Lei, Jiaming Guo, Guanhua He, Xishan Zhang, Rui Zhang, Shaohui Peng, Shaoli Liu, Tianshi Chen

Challenge:	Generating long-term texts using artificial intelligence has always been a challenge . however, the generated novels exhibit poor logical coherence and appeal in their plots and deficiencies in character and event depiction, ultimately compromising the overall narrative quality.
Approach:	They propose a method for extracting excelsior and expanding from novel data to generate arbitrarily long novels using large language models.
Outcome:	The proposed method produces high-quality long-form novels with a high level of logical coherence and appeal despite the use of large language models.

DCIS: Efficient Length Extrapolation of LLMs via Divide-and-Conquer Scaling Factor Search (2025.emnlp-main)

Copied to clipboard

Lei Yang, Shaoyang Xu, Jianxiang Peng, Shaolin Zhu, Deyi Xiong

DocEE-zh: A Fine-grained Benchmark for Chinese Document-level Event Extraction (2024.findings-emnlp)

Challenge:	Existing frameworks for large language models with context length limitations are suboptimal for initialization and fine-tuning.
Approach:	They propose a RoPE-based fine-tuning framework that strategically determines the best scaling factors for LLMs by a Divide-and-Conquer Incremental Search algorithm.
Outcome:	The proposed framework mitigates performance decay at extended target lengths and can perform effectively without fine-tuning.

Copied to clipboard

Minghui Liu, MeiHan Tong, Yangda Peng, Lei Hou, Juanzi Li, Bin Xu

Do Pre-trained Models Benefit Knowledge Graph Completion? A Reliable Evaluation and a Reasonable Approach (2022.findings-acl)

Challenge:	Chinese document-level event extraction is still largely unexplored.
Approach:	They propose a Chinese document-level event extraction dataset with over 36,000 events and 210,000 arguments.
Outcome:	The proposed dataset includes over 36,000 events and more than 210,000 arguments . it is an extension of the DocEE dataset, utilizing the same event schema and annotated by human experts.

Copied to clipboard

Xin Lv, Yankai Lin, Yixin Cao, Lei Hou, Juanzi Li, Zhiyuan Liu, Peng Li, Jie Zhou

Pre-training Distillation for Large Language Models: A Design Space Exploration (2025.acl-long)

Challenge:	Pre-trained language models capture factual knowledge from massive texts . but they are still quite behind the SOTA KGC models in terms of performance .
Approach:	They propose to use open-world assumption to evaluate PLM-based knowledge graph completion models . they propose to convert each triple and its support information into natural prompt sentences .
Outcome:	The proposed model is more accurate under the open-world assumption (OWA) this setting manual checks the correctness of knowledge that is not in KGs.

Copied to clipboard

Hao Peng, Xin Lv, Yushi Bai, Zijun Yao, Jiajie Zhang, Lei Hou, Juanzi Li

GroupRank: A Groupwise Paradigm for Effective and Efficient Passage Reranking with LLMs (2026.findings-acl)

Challenge:	Knowledge distillation (KD) aims to transfer knowledge from a large teacher model to a smaller student model for model compression.
Approach:	They extend knowledge distillation to the pre-training phase of large language models . they first conduct an experiment using a teacher LLM to distill a 1.9B student LLM .
Outcome:	The proposed model can be used to distill a 1.9B student model using a teacher LLM.

Copied to clipboard

Meixiu Long, Duolin Sun, Dan Yang, Yihan Jiao, Lei Liu, Jiahai Wang, Binbin Hu, Yue Shen, Jie Feng, Zhehao Tan, Junjie Wang, Lianzhen Zhong, Jian Wang, Peng Wei, Jinjie Gu

CAFE: Retrieval Head-based Coarse-to-Fine Information Seeking to Enhance Multi-Document QA Capability (2025.emnlp-main)

Challenge:	Existing rankers excel in lexical-matching scenarios, while they struggle with complex queries requiring deep reasoning.
Approach:	They propose a new paradigm that balances flexibility and context awareness to unlock the full potential of groupwise reranking.
Outcome:	The proposed approach achieves a state-of-the-art 65.2 NDCG@10 on BRIGHT and surpasses baselines by 2.1 points on R2MED while delivering a 6.4 inference speedup.

Copied to clipboard

Han Peng, Jinhao Jiang, Zican Dong, Xin Zhao, Lei Fang

Question Answering

Challenge:	Existing methods to extend context length of Large Language Models (LLMs) still struggle with retrieval and reasoning in long context inputs.
Approach:	They propose a coarse-to-fine method to enhance multi-document question-answering capacities by removing background and distracting documents.
Outcome:	Experiments show that CAFE outperforms baseline methods on multiple documents.

Dense Procedure Captioning in Narrated Instructional Videos (P19-1)

Copied to clipboard

Botian Shi, Lei Ji, Yaobo Liang, Nan Duan, Peng Chen, Zhendong Niu, Ming Zhou

Multimodality

Challenge:	Existing models for video dense captioning learn video segments and generate captions without considering transcripts.
Approach:	They propose a model to generate procedure captions from narrated instructional videos . they extract procedures by a cross-modality module and generate captions by encoding video frames and transcripts within each extracted procedure.
Outcome:	The proposed model can extract procedures from narrated instructional videos and generate procedure captions by encoding video frames and transcripts.

OneGen: Efficient One-Pass Unified Generation and Retrieval for LLMs (2024.findings-emnlp)

Copied to clipboard

Jintian Zhang, Cheng Peng, Mengshu Sun, Xiang Chen, Lei Liang, Zhiqiang Zhang, Jun Zhou, Huajun Chen, Ningyu Zhang

Green & Sustainable NLP

Challenge:	Recent advances in Large Language Models (LLMs) have significantly enhanced the generative capabilities for various NLP tasks, but they still suffer from hallucinations due to their exclusive reliance on parametric knowledge.
Approach:	They propose a framework that integrates retrieval tokens generated autoregressively into a single LLM to handle both tasks simultaneously in a unified forward pass.
Outcome:	The proposed framework bridges the traditionally separate training approaches for generation and retrieval by incorporating retrieval tokens generated autoregressively.

Automate Strategy Finding with LLM in Quant Investment (2025.findings-emnlp)

Copied to clipboard

Zhizhuo Kou, Holam Yu, Junyu Luo, Jingshu Peng, Xujia Li, Chengzhong Liu, Juntao Dai, Lei Chen, Sirui Han, Yike Guo

Half-S: Halving the Scale for Near-Lossless 4-Bit LLM Training (2026.findings-acl)

Challenge:	Experimental results demonstrate robust performance of the strategy in Chinese & US market regimes compared to established benchmarks.
Approach:	They propose a framework leveraging Large Language Models within a risk-aware multi-agent system for automate strategy finding in quantitative finance.
Outcome:	The proposed framework outperforms all benchmarks in Chinese & US market regimes with 53.17% cumulative return on SSE50.

Copied to clipboard

Jinyang Du, Ruihao Gong, Linghan Ai, Zining Wang, Yunke Peng, Yao Wang, Lei Yan, null Wxuefei, Yaoyuan Wang, Jinyang Guo, Dahua Lin, Xianglong Liu

Green & Sustainable NLP

Challenge:	Existing 4-bit training pipelines rely on max-scaling, which causes representation collapse . despite this, there are limitations in the accuracy of 4-bit LLM training .
Approach:	They propose a scaling strategy that uses half-scaling as a hardware-friendly default . they propose fp4 support that allows for a faster scaling of large language models .
Outcome:	The proposed scaling strategy narrows the gap between theoretical optimum and BF16 while maintaining the efficiency benefits of 4-bit training.

Dynamic Voting for Efficient Reasoning in Large Language Models (2023.findings-emnlp)

Copied to clipboard

Mingfeng Xue, Dayiheng Liu, Wenqiang Lei, Xingzhang Ren, Baosong Yang, Jun Xie, Yidan Zhang, Dezhong Peng, Jiancheng Lv

Aspect Is Not You Need: No-aspect Differential Sentiment Framework for Aspect-based Sentiment Analysis (2022.naacl-main)

Challenge:	Multi-path voting methods generate multiple reasoning paths for each problem, causing factual errors and illusion generation.
Approach:	They propose a multi-path voting technique that effectively reduces the number of reasoning paths during multi-path voting while preserving accuracies.
Outcome:	The proposed method outperforms Self-consistency using 24.7% of the number of paths on the LetterConcat task.

Copied to clipboard

Jiahao Cao, Rui Liu, Huailiang Peng, Lei Jiang, Xu Bai

Sentiment Analysis

Challenge:	Existing approaches to classify aspects with aspect sentiment bias are hard to find .
Approach:	They propose a no-aspect differential sentiment framework for the ABSA task that eliminates aspect sentiment bias and uses differential sentiment loss instead of cross-entropy loss to better classify the sentiments.
Outcome:	The proposed framework can be combined with almost all traditional ABSA methods.

From Mimicking to Integrating: Knowledge Integration for Pre-Trained Language Models (2022.findings-emnlp)

Copied to clipboard

Lei Li, Yankai Lin, Xuancheng Ren, Guangxiang Zhao, Peng Li, Jie Zhou, Xu Sun

A Scalable Multi-LLM Collaboration System with Retrieval-based Selection and Exploration-Exploitation-Driven Enhancement (2026.acl-long)

Challenge:	Existing models for natural language processing (NLP) are fine-tuned and released for research and deployments.
Approach:	They propose a PLM reuse paradigm that merges teacher-PLM knowledge into a student model.
Outcome:	The proposed paradigm can reduce the computational cost and environmental side-effects of retraining the PLM from scratch.

Copied to clipboard

Shengji Tang, Jianjian Cao, Weihao Lin, Jiale Hong, Bo Zhang, Shuyue Hu, Lei Bai, Tao Chen, Wanli Ouyang, Peng Ye

VisKoP: Visual Knowledge oriented Programming for Interactive Knowledge Base Question Answering (2023.acl-demo)

Challenge:	Existing multi-LLM collaboration systems often encounter scalability challenges when integrating new LLMs and tasks.
Approach:	They propose a Scalable Multi-LLM Collaboration System to coordinate multiple open-source LLMs.
Outcome:	The proposed system outperforms prevailing closed-source LLMs on eight mainstream benchmarks on multiple tasks.

Copied to clipboard

Zijun Yao, Yuanyong Chen, Xin Lv, Shulin Cao, Amy Xin, Jifan Yu, Hailong Jin, Jianjun Xu, Peng Zhang, Lei Hou, Juanzi Li

Multimodality

Challenge:	Existing knowledge base question answering systems that parse natural language questions into knowledge oriented program language (KoPL) .
Approach:	They propose a knowledge base question answering system that integrates human into the loop to edit and debug queries.
Outcome:	The proposed system can debug and edit knowledge base questions on a million-entity-level . it provides auto-completion for its knowledge base schema and user interaction can fix a large portion of wrong KoPL programs to acquire the correct answer.

MAVEN-FACT: A Large-scale Event Factuality Detection Dataset (2024.findings-emnlp)

Copied to clipboard

Chunyang Li, Hao Peng, Xiaozhi Wang, Yunjia Qi, Lei Hou, Bin Xu, Juanzi Li

Neeko: Leveraging Dynamic LoRA for Efficient Multi-Character Role-Playing Agent (2024.emnlp-main)

Challenge:	Event factuality detection is under-explored due to the lack of high-quality large-scale data . efd is a subfield of event understanding, which aims to determine the factuity of textual events.
Approach:	They propose a large-scale EFD dataset with factuality annotations of 112,276 events . they find that adopting event arguments and relations helps in event factuity detection .
Outcome:	The proposed dataset includes factuality annotations of 112,276 events . it is the largest EFD dataset and is challenging for fine-tuned models and large language models .

Copied to clipboard

Xiaoyan Yu, Tongxu Luo, Yifan Wei, Fangyu Lei, Yiming Huang, Hao Peng, Liehuang Zhu

Green & Sustainable NLP

Challenge:	Large Language Models (LLMs) have revolutionized open-domain dialogue agents but face challenges in multi-character role-playing (MCRP) scenarios.
Approach:	They propose a framework for efficient multi-character role-playing that employs a dynamic low-rank adapter strategy and distinct LoRA blocks for each character.
Outcome:	Neeko employs a dynamic low-rank adapter (LoRA) strategy, enabling it to adapt seamlessly to diverse characters.

Agentic Reward Modeling: Integrating Human Preferences with Verifiable Correctness Signals for Reliable Reward Systems (2025.acl-long)

Copied to clipboard

Hao Peng, Yunjia Qi, Xiaozhi Wang, Zijun Yao, Bin Xu, Lei Hou, Juanzi Li

Challenge:	Existing reward models focus on human preferences, neglecting verifiable correctness signals.
Approach:	They propose a reward system that combines human preference rewards with verifiable correctness signals to provide reliable rewards.
Outcome:	The proposed reward agent significantly outperforms vanilla reward models on benchmarks and inference-time best-of-n searches on real-world tasks.

Learning Fine-Grained Grounded Citations for Attributed Large Language Models (2024.findings-acl)

Copied to clipboard

Lei Huang, Xiaocheng Feng, Weitao Ma, Yuxuan Gu, Weihong Zhong, Xiachong Feng, Weijiang Yu, Weihua Peng, Duyu Tang, Dandan Tu, Bing Qin

MMD-ERE: Multi-Agent Multi-Sided Debate for Event Relation Extraction (2025.coling-main)

Challenge:	despite impressive performance, large language models still struggle with hallucinations . current approaches suffer from suboptimal citation quality due to reliance on in-context learning .
Approach:	They propose a framework that teaches large language models to generate fine-grained citations.
Outcome:	The proposed framework outperforms all baselines on the ALCE benchmark and achieves an average improvement of 14.21% in citation quality.

Copied to clipboard

Yong Guan, Hao Peng, Lei Hou, Juanzi Li

Beyond Layout Embedding: Layout Attention with Gaussian Biases for Structured Document Understanding (2023.findings-emnlp)

Challenge:	Existing research indicates that LLMs can be overconfident and stubborn.
Approach:	They propose a multi-agent multi-sided debate approach for event relation extraction which explores the understanding of event relations between different participants before and after the debate.
Outcome:	The proposed approach outperforms established baselines on various ERE tasks and LLMs.

Copied to clipboard

Xi Zhu, Xue Han, Shuyuan Peng, Shuo Lei, Chao Deng, Junlan Feng

Representation Learning

Challenge:	Existing methods for encoding layout information rely on millions of learnable parameters . polar coordinates provide superior choice for layout modeling, study finds .
Approach:	They propose to model layout attention with Gaussian biases by feeding polar coordinates into 2-D Gausssian kernels.
Outcome:	The proposed model improves on three widely used benchmarks.

Dynamic Knowledge Distillation for Pre-trained Language Models (2021.emnlp-main)

Copied to clipboard

Lei Li, Yankai Lin, Shuhuai Ren, Peng Li, Jie Zhou, Xu Sun

CascadeBERT: Accelerating Inference of Pre-trained Language Models via Calibrated Complete Models Cascade (2021.findings-emnlp)

Challenge:	Existing methods conduct knowledge distillation statically, e.g., student model aligns output distribution to teacher model on pre-defined training dataset.
Approach:	They propose a dynamic knowledge distillation that empowers the student to adjust the learning procedure according to its competency . they find it is promising and provide discussions on potential future directions towards more efficient methods .
Outcome:	The proposed method can boost student model performance while accelerating training . the proposed method reduces memory usage and accelerates model inference .

Copied to clipboard

Lei Li, Yankai Lin, Deli Chen, Shuhuai Ren, Peng Li, Jie Zhou, Xu Sun