Papers by Jie Peng

Contrastive Preference Learning for Neural Machine Translation (2024.findings-naacl)

Challenge:	Experimental results show that pre-trained language model GPT2 can generate better continuations by learning to generate the in the fine-tuning stage.
Approach:	They conduct experiments on an English essay dataset using Chinese-GPT2 . they find that the model can generate better continuations by learning to generate the in the fine-tuning stage.
Outcome:	The pre-trained language model GPT2 can generate better continuations by learning to generate the in the fine-tuning stage.

Copied to clipboard

Jianfei He, Shichao Sun, Sen Peng, Jie Xu, Xiaohua Jia, Wenjie Li

Machine Translation

Challenge:	Existing discrepancies between token-level objective and overall sequence-level quality of a model are causing exposure bias and other issues in NMT.
Approach:	They propose a contrastive preference model that integrates an indicator function to fine-tune a pre-trained model in Neural Machine Translation.
Outcome:	The proposed model outperforms the traditional Plackett-Luce model on three language pairs and also outperFORMs token-level and sequence-level baseline models.

WebClipper: Efficient Evolution of Web Agents with Graph-based Trajectory Pruning (2026.acl-long)

Copied to clipboard

Junjie Wang, Zequn Xie, Dan Yang, Jie Feng, Yue Shen, Duolin Sun, Meixiu Long, Yihan Jiao, Zhehao Tan, Jian Wang, Peng Wei, Jinjie Gu

Knowledge Inheritance for Pre-trained Language Models (2022.naacl-main)

Challenge:	Open-source web agents rely on long tool-call trajectories with cyclic reasoning loops and exploration of unproductive branches.
Approach:	They propose a framework that compresses web agent trajectories via graph-based pruning.
Outcome:	The proposed framework reduces tool-call rounds by 20% while improving accuracy and efficiency while maintaining the same level of performance as existing models.

Copied to clipboard

Yujia Qin, Yankai Lin, Jing Yi, Jiajie Zhang, Xu Han, Zhengyan Zhang, Yusheng Su, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou

MAVEN: A Massive General Domain Event Detection Dataset (2020.emnlp-main)

Challenge:	Existing large-scale pre-trained language models are mainly trained from scratch individually, ignoring that many well-taught PLMs are available.
Approach:	They propose a pre-training framework called knowledge inheritance and propose auxiliary supervision to efficiently learn larger PLMs.
Outcome:	The proposed framework can be used to train large-scale language models with huge parameters and a large dataset can be adapted to domain adaptation and knowledge transfer.

Copied to clipboard

Xiaozhi Wang, Ziqi Wang, Xu Han, Wangyi Jiang, Rong Han, Zhiyuan Liu, Juanzi Li, Peng Li, Yankai Lin, Jie Zhou

Are Intermediate Layers and Labels Really Necessary? A General Language Model Distillation Method (2023.findings-acl)

Challenge:	Existing datasets exhibit data scarcity and limited coverage of general-domain events.
Approach:	They present a MAssive eVENt detection dataset which contains 4,480 Wikipedia documents and 168 event types.
Outcome:	The proposed dataset shows that existing methods cannot achieve promising results on the small datasets.

Copied to clipboard

Shicheng Tan, Weng Lam Tam, Yuanchun Wang, Wenwen Gong, Shu Zhao, Peng Zhang, Jie Tang

MAVEN-ARG: Completing the Puzzle of All-in-One Event Understanding Dataset with Event Argument Annotation (2024.acl-long)

Challenge:	Existing knowledge distillation methods rely on intermediate layer features and golden labels, which require aligned model architecture and labeled data respectively.
Approach:	They propose a general language model distillation method that performs two-stage word prediction distillation and vocabulary compression, which is simple and shows extremely strong performance.
Outcome:	The proposed method outperforms 25 state-of-the-art methods on the SuperGLUE benchmark, achieving an average score that surpasses the best method by 3%.

Copied to clipboard

Xiaozhi Wang, Hao Peng, Yong Guan, Kaisheng Zeng, Jianhui Chen, Lei Hou, Xu Han, Yankai Lin, Zhiyuan Liu, Ruobing Xie, Jie Zhou, Juanzi Li

Towards Fine-grained Text Sentiment Transfer (P19-1)

Challenge:	Existing datasets for event understanding have limited coverage due to complexity of tasks.
Approach:	They propose a dataset that augments MAVEN datasets with event argument annotations . they propose 98,591 events and 290,613 arguments obtained with laborious human annotation .
Outcome:	The proposed dataset is the first all-in-one dataset supporting event detection, event argument extraction, and event relation extraction.

Copied to clipboard

Fuli Luo, Peng Li, Pengcheng Yang, Jie Zhou, Yutong Tan, Baobao Chang, Zhifang Sui, Xu Sun

Sentiment Analysis

Challenge:	Existing methods for fine-grained text sentiment transfer only reverse the sentiment polarity of text, but they lack a robust and parallel learning algorithm.
Approach:	They propose a novel fine-grained text sentiment transfer task that revises a sequence to satisfy a given sentiment intensity while preserving the original semantic content.
Outcome:	The proposed model outperforms existing methods by a large margin in automatic evaluation and human evaluation.

More Data, More Relations, More Context and More Openness: A Review and Outlook for Relation Extraction (2020.aacl-main)

Copied to clipboard

Xu Han, Tianyu Gao, Yankai Lin, Hao Peng, Yaoliang Yang, Chaojun Xiao, Zhiyuan Liu, Peng Li, Jie Zhou, Maosong Sun

UQ-Merge: Uncertainty Guided Multimodal Large Language Model Merging (2025.findings-acl)

Challenge:	Existing methods for extracting relational facts from text have been successful . but with explosion of Web text, human knowledge is increasing drastically .
Approach:	They propose to improve relation extraction methods to extract relational facts from text . they analyze existing methods and show promising directions towards more powerful RE .
Outcome:	The proposed methods can extract relational facts from text, but they are still lacking in the current field.

Copied to clipboard

Huaizhi Qu, Xinyu Zhao, Jie Peng, Kwonjoon Lee, Behzad Dariush, Tianlong Chen

PaCoRe: Learning to Scale Test-Time Compute with Parallel Coordinated Reasoning (2026.acl-long)

Challenge:	Existing models merging methods often lead to suboptimal performance due to harmful models . et al., 2018; 59: 59-64.
Approach:	They propose an uncertainty-guided MLLM merging algorithm that integrates models into a single MLML.
Outcome:	The proposed algorithm improves on held-in and held-out vision-language benchmarks.

Copied to clipboard

Jingcheng Hu, Yinmin Zhang, Shijie Shang, Xiaobo Yang, Yue Peng, Zhewei Huang, Hebin Zhou, Xin Wu, Jie Cheng, Fanqi Wan, Xiangwen Kong, Chengyuan Yao, Kaiwen Yan, Ailin Huang, Hongyu Zhou, Qi Han, Zheng Ge, Xiangyu Zhang, Heung-Yeung Shum

ℛ3: Advertisement Compliance ℛectification via Group-ℛelative Experience Extractor and Curriculum ℛeinforcement (2026.acl-industry)

Challenge:	Parallel Coordinated Reasoning (PaCoRe) overcomes a central limitation of contemporary language models: their inability to scale test-time compute (TTC) far beyond sequential reasoning under a fixed context window.
Approach:	They propose a training-and-inference framework to overcome a central limitation of language models: their inability to scale test-time compute (TTC) under a fixed context window.
Outcome:	The proposed model scales to multi-million-token effective TTC without exceeding context limits.

Copied to clipboard

Yuan Chen, Zhenyu Hu, Mengge Xue, Cao Te, Liqun Liu, Peng Shu, Huan Yu, Jie Jiang

CTRLEval: An Unsupervised Reference-Free Metric for Evaluating Controlled Text Generation (2022.acl-long)

Challenge:	Existing methods of content moderation are infeasible due to over-editing and compromise the advertiser’s original semantic intent.
Approach:	They propose a framework to harmonize compliance with original intent preservation that integrates a data-driven framework and a curriculum to enforce compliance while maximizing semantic consistency.
Outcome:	The proposed framework outperforms state-of-the-art baselines on industrial datasets and on online A/B testing on industrial video.

Copied to clipboard

Pei Ke, Hao Zhou, Yankai Lin, Peng Li, Jie Zhou, Xiaoyan Zhu, Minlie Huang

Low-Resource NLP

Challenge:	Existing reference-free metrics have obvious limitations for evaluating controlled text generation models.
Approach:	They propose an unsupervised reference-free metric which evaluates controlled text generation from different aspects by formulating each aspect into multiple text infilling tasks.
Outcome:	The proposed metric has higher correlations with human judgments while obtaining better generalization of evaluating generated texts from different models and with different qualities.

HIRAG: Hierarchical-Thought Instruction-Tuning Retrieval-Augmented Generation (2025.findings-emnlp)

Copied to clipboard

Yihan Jiao, Zhehao Tan, Dan Yang, Duolin Sun, Jie Feng, Yue Shen, Jian Wang, Peng Wei

Neural Gibbs Sampling for Joint Event Argument Extraction (2020.aacl-main)

Challenge:	In-depth research on the specific capabilities needed by the RAG generation model is lacking, leading to inconsistent document quality and retrieval system imperfections.
Approach:	They propose that RAG models should possess three progressively hierarchical abilities: (1) Filtering: the ability to select relevant information; (2) Combination: the capability to combine semantic information across paragraphs; (3) RAG-specific reasoning: the capacity to further process external knowledge using internal knowledge.
Outcome:	Experiments show that the proposed method significantly improves the model’s open-book examination capability on datasets such as RGB, PopQA, MuSiQue, HotpotQA, and PubmedQA.

Copied to clipboard

Xiaozhi Wang, Shengyu Jia, Xu Han, Zhiyuan Liu, Juanzi Li, Peng Li, Jie Zhou

LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitasks (2025.acl-long)

Challenge:	Existing methods for event argument extraction cannot adequately model the correlation between event arguments and their roles.
Approach:	They propose a Bayesian model to jointly extract event arguments using Gibbs sampling . they train two neural networks to model prior distribution and conditional distribution over event arguments .
Outcome:	The proposed model can achieve comparable results to existing methods on two widely-used datasets.

Copied to clipboard

Yushi Bai, Shangqing Tu, Jiajie Zhang, Hao Peng, Xiaozhi Wang, Xin Lv, Shulin Cao, Jiazheng Xu, Lei Hou, Yuxiao Dong, Jie Tang, Juanzi Li

Unifying Discrete and Continuous Representations for Unsupervised Paraphrase Generation (2023.emnlp-main)

Challenge:	Existing benchmarks on longcontext large language models fail to reflect their deep understanding capabilities across diverse tasks.
Approach:	They propose a benchmark to assess the ability of long-context large language models to handle long-text problems.
Outcome:	The proposed model achieves 50.1% accuracy when directly answering the questions . human experts achieve only 53.7% accuracy under a 15-minute time constraint .

Copied to clipboard

Mingfeng Xue, Dayiheng Liu, Wenqiang Lei, Jie Fu, Jian Lan, Mei Li, Baosong Yang, Jun Xie, Yidan Zhang, Dezhong Peng, Jiancheng Lv

Text Generation

Challenge:	Existing unsupervised paraphrase generation methods require large-scale, manually annotated paraphrase datasets, which are labor-intensive to build.
Approach:	They propose a self-supervised pseudo-data construction method that generates diverse pseudo-paraphrases in distinct surface structures for a given sentence.
Outcome:	The proposed method generates diverse pseudo-paraphrases in distinct surface structures for a given sentence.

Disentangle-based Continual Graph Representation Learning (2020.emnlp-main)

Copied to clipboard

Xiaoyu Kou, Yankai Lin, Shaobo Liu, Peng Li, Jie Zhou, Yan Zhang

ReFT: Reasoning with Reinforced Fine-Tuning (2024.acl-long)

Challenge:	Existing graph embedding methods overlook streaming nature of incoming data in real-world applications.
Approach:	They propose a disentangle-based continual graph representation learning framework inspired by the human’s ability to learn procedural knowledge.
Outcome:	The proposed framework outperforms state-of-the-art continual graph representation learning framework and alleviate catastrophic forgetting problem.

Copied to clipboard

Luong Trung, Xinbo Zhang, Zhanming Jie, Peng Sun, Xiaoran Jin, Hang Li

CLEVE: Contrastive Pre-training for Event Extraction (2021.acl-long)

Challenge:	Existing approaches to improve the generalization of large language models are using Supervised Fine-Tuning (SFT) this approach does not show sufficient generalization ability because it only relies on the given CoT data.
Approach:	They propose to use Chain-of-Thought annotations to train Large Language Models using supervised fine-tuning to improve generalization.
Outcome:	The proposed approach outperforms SFT on GSM8K, MathQA, and SVAMP datasets and shows a superior generalization ability.

Copied to clipboard

Ziqi Wang, Xiaozhi Wang, Xu Han, Yankai Lin, Lei Hou, Zhiyuan Liu, Peng Li, Juanzi Li, Jie Zhou

Reasoning with Ontology Graph: Toward Type-Constrained Knowledge Graph Question Answering (2026.acl-long)

Challenge:	Existing EE methods do not model event characteristics from large unsupervised data.
Approach:	They propose a contrastive pre-training framework for event extraction to better learn event knowledge from large unsupervised data and their semantic structures.
Outcome:	The proposed framework improves on ACE 2005 and MAVEN datasets on event extraction tasks.

Copied to clipboard

Yongxue Shan, Jie Peng, Zixuan Dong, Fei Hu, Xiaodong Wang

Knowledge Representation

Challenge:	Existing knowledge graph question answering methods rely on LLM-induced type systems with inconsistent granularity or perform multi-hop reasoning without explicit target-type constraints.
Approach:	They propose a type-constrained knowledge graph question answering framework that reasons over a relation-centric ontology graph.
Outcome:	The proposed framework achieves state-of-the-art and produces ontology-grounded reasoning chains with substantial Hit@1 gains.

HOSMEL: A Hot-Swappable Modularized Entity Linking Toolkit for Chinese (2022.acl-demo)

Copied to clipboard

Daniel Zhang-li, Jing Zhang, Jifan Yu, Xiaokang Zhang, Peng Zhang, Jie Tang, Juanzi Li

Knowledge Representation

Challenge:	Existing studies have explored the use of entity linking (EL) in downstream tasks.
Approach:	They propose a modularized entity linking toolkit for easy task adaptation.
Outcome:	The proposed toolkit achieves significantly better accuracy and less time and spaceconsumption than existing methods.

Layer-Level Self-Exposure and Patch: Affirmative Token Mitigation for Jailbreak Attack Defense (2025.naacl-long)

Copied to clipboard

Yang Ouyang, Hengrui Gu, Shuhang Lin, Wenyue Hua, Jie Peng, Bhavya Kailkhura, Meijun Gao, Tianlong Chen, Kaixiong Zhou

FewRel 2.0: Towards More Challenging Few-Shot Relation Classification (D19-1)

Challenge:	Existing methods to defend against jailbreak attacks exploit vulnerabilities to elicit unintended or harmful outputs.
Approach:	They propose a method to defend against jailbreak attacks by patching specific layers within large language models through self-augmented datasets.
Outcome:	The proposed approach reduces harmfulness and attack success rate of jailbreak attacks without compromising utility for benign queries compared to previous methods.

Copied to clipboard

Tianyu Gao, Xu Han, Hao Zhu, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou

GRNFormer: A Biologically-Guided Framework for Integrating Gene Regulatory Networks into RNA Foundation Models (2025.findings-acl)

Challenge:	Few-shot domain adaptation and NOTA detection are two real-world challenges for few-shot relation classification models.
Approach:	They propose a task to investigate two aspects of few-shot relation classification models . they build upon the FewRel dataset by adding a new test set in a different domain .
Outcome:	The proposed task can evaluate few-shot domain adaptation and few- shot none-of-the-above detection on a new domain and NOTA relation choice.

Copied to clipboard

Mufan Qiu, Xinyu Hu, Fengwei Zhan, Sukwon Yun, Jie Peng, Ruichen Zhang, Bhavya Kailkhura, Jiekun Yang, Tianlong Chen

MAVEN-ERE: A Unified Large-scale Dataset for Event Coreference, Temporal, Causal, and Subevent Relation Extraction (2022.emnlp-main)

Challenge:	Foundation models for single-cell RNA sequencing ignore biological prior knowledge encoded in gene regulatory relationships and fail to leverage multi-omics signals.
Approach:	They propose a framework that integrates multi-scale gene regulatory networks into RNA foundation model training.
Outcome:	The proposed framework improves on state-of-the-art models on three downstream tasks . it integrates multi-scale gene regulatory networks (GRNs) from multi-omics data into training .

Copied to clipboard

Xiaozhi Wang, Yulin Chen, Ning Ding, Hao Peng, Zimu Wang, Yankai Lin, Xu Han, Lei Hou, Juanzi Li, Zhiyuan Liu, Peng Li, Jie Zhou

MoEfication: Transformer Feed-forward Layers are Mixtures of Experts (2022.findings-acl)

Challenge:	Existing datasets only cover limited relation types at once, which prevents models from taking full advantage of relation interactions.
Approach:	They construct a large-scale human-annotated ERE dataset with improved annotation schemes to address these drawbacks.
Outcome:	The proposed dataset is larger than existing datasets of all the ERE tasks by at least an order of magnitude.

Copied to clipboard

Zhengyan Zhang, Yankai Lin, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou

Unsupervised Knowledge Selection for Dialogue Generation (2021.findings-acl)

Challenge:	Recent work has shown that feed-forward networks (FFNs) in pre-trained Transformers are a key component, storing various linguistic and factual knowledge.
Approach:	They propose to convert a model into its MoE version with the same parameters and build expert routers to decide which experts will be used for each input.
Outcome:	The proposed model can use 10% to 30% of FFN parameters while maintaining over 95% original performance.

Copied to clipboard

Xiuyi Chen, Feilong Chen, Fandong Meng, Peng Li, Jie Zhou

Dialogue Systems & Conversational Agents

Challenge:	Existing knowledge selection tasks require the preidentified knowledge to generate informative dialogues.
Approach:	They propose a novel method to supervise knowledge selection when the gold knowledge label is unknown by obtaining an oracle knowledge label via distant supervision and leverage knowledge distillation to alleviate the noisy labeling problem of distant supervision.
Outcome:	The proposed method outperforms strong supervised baselines on two knowledge-grounded dialogue datasets and generates more informative responses.

Unsupervised Dependency Graph Network (2022.acl-long)

Copied to clipboard

Yikang Shen, Shawn Tan, Alessandro Sordoni, Peng Li, Jie Zhou, Aaron Courville

Learning Flexible Large Multimodal Models with Arbitrary Modality Combinations (2026.findings-acl)

Challenge:	Recent work has identified properties of pretrained self-attention models that mirror those of dependency parse structures.
Approach:	They propose a model that encourages attention heads to model different dependency relations from raw corpora and a masked language modeling task.
Outcome:	The proposed model can induce dependency structures from raw corpora and the masked language modeling task without gold POS tags and any external information.

Copied to clipboard

Xinyu Zhao, Kangqi Ni, Jie Peng, Ang Li, Tianlong Chen

Dialogue is Better Than Monologue: Instructing Meidcal LLMs via Strategic Conversations (2026.findings-eacl)

Challenge:	Multimodal Large Language Models (MLLMs) have potential for cross-modal understanding . but extending MLLM to handle diverse modalities introduces two challenges .
Approach:	They propose a dual-stage compression mechanism to reduce the number of modality tokens per modality and condense it into a single, compact token sequence.
Outcome:	Experiments show that Flex-M3 outperforms its counterpart trained on only full-modality data.

Copied to clipboard

Zijie Liu, Xinyu Zhao, Jie Peng, Jinhao Duan, Zhuangdi Zhu, Qingyu Chen, Kaidi Xu, Xia Hu, Tianlong Chen

Natural Language Interfaces

Challenge:	Existing tuning methods for medical AI models are monologue-based . existing benchmarks are based on licensing exams or research articles .
Approach:	They propose a benchmark to expose limitations of monologue-based tuning for medical AI models . they use a large dialogue dataset to capture stepwise diagnostic reasoning .
Outcome:	The proposed model outperforms monologue-tuned models on a medical question answering task and improves accuracy on standard medical QA benchmarks.

On Transferability of Prompt Tuning for Natural Language Processing (2022.naacl-main)

Copied to clipboard

Yusheng Su, Xiaozhi Wang, Yujia Qin, Chi-Min Chan, Yankai Lin, Huadong Wang, Kaiyue Wen, Zhiyuan Liu, Peng Li, Juanzi Li, Lei Hou, Maosong Sun, Jie Zhou

Plug-and-Play Knowledge Injection for Pre-trained Language Models (2023.acl-long)

Challenge:	Pre-trained language models (PLMs) can achieve comparable performance to full-parameter fine-tuning by tuning a few soft prompts, but require much more training time than fine-timing.
Approach:	They empirically investigate the transferability of soft prompts across different downstream tasks and PLMs to determine what decides prompt transferability.
Outcome:	The proposed method can achieve comparable performance to full-parameter fine-tuning by tuning a few soft prompts, but requires much more training time than fine-timing.

Copied to clipboard

Zhengyan Zhang, Zhiyuan Zeng, Yankai Lin, Huadong Wang, Deming Ye, Chaojun Xiao, Xu Han, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou

Towards Robust Visual Question Answering: Making the Most of Biased Samples via Contrastive Learning (2022.findings-emnlp)

Challenge:	Existing knowledge injection methods are not suitable for enhancing pre-trained language models with external knowledge bases.
Approach:	They propose a plug-and-play knowledge injection method where knowledge bases are injected into frozen existing downstream models by a knowledge plugin.
Outcome:	The proposed method improves the performance of knowledge injection on knowledge-driven tasks while keeping model parameters frozen.

Copied to clipboard

Qingyi Si, Yuanxin Liu, Fandong Meng, Zheng Lin, Peng Fu, Yanan Cao, Weiping Wang, Jie Zhou

RAP: Robustness-Aware Perturbations for Defending against Backdoor Attacks on NLP Models (2021.emnlp-main)

Challenge:	Recent studies have shown that biased samples can be brittle for VQA models . however, the improvements on OOD data severely sacrifice the performance on the in-distribution (ID) data.
Approach:	They propose a contrastive learning approach that exploits biased samples for unbiased information that contributes to reasoning.
Outcome:	The proposed method achieves competitive performance on the OOD dataset while maintaining robustness on the ID dataset.

Copied to clipboard

Wenkai Yang, Yankai Lin, Peng Li, Jie Zhou, Xu Sun

Robustness in NLP

Challenge:	Backdoor attacks are a serious threat to the safety of reusing deep neural networks (DNNs).
Approach:	They propose an efficient online defense mechanism based on robustness-aware perturbations to distinguish poisoned and clean samples to defend against backdoor attacks on natural language processing models.
Outcome:	The proposed method achieves better defending performance and lower computational costs than existing defense methods.

NumNet: Machine Reading Comprehension with Numerical Reasoning (D19-1)

Copied to clipboard

Qiu Ran, Yankai Lin, Peng Li, Jie Zhou, Zhiyuan Liu

Towards Faithful Industrial RAG: A Reinforced Co-adaptation Framework for Advertising QA (2026.acl-industry)

Challenge:	Existing numerical MRC models are weak in numerical reasoning, such as addition, subtraction, sorting and counting.
Approach:	They propose a numerical MRC model that integrates numerical reasoning into existing MRC models and achieves an EM-score of 64.56% on the DROP dataset.
Outcome:	The proposed model outperforms all existing machine reading comprehension models by considering the numerical relations among numbers on the DROP dataset.

Copied to clipboard

Wenwei Li, Ming Xu, Tianle Xia, Lingxiang Hu, Yiding Sun, Linfang Shang, Liqun Liu, Peng Shu, Huan Yu, Jie Jiang

Key Fact as Pivot: A Two-Stage Model for Low Resource Table-to-Text Generation (P19-1)

Challenge:	Existing methods for QA in industrial environments are inherently relational and often updated.
Approach:	They propose a framework that optimizes retrieval and generation through two components: Graph-aware Retrieval and evidence-constrained reinforcement learning.
Outcome:	Experiments on an internal advertising QA dataset show consistent gains across expert-judged dimensions including accuracy, completeness, safety, and URL validity.

Copied to clipboard

Shuming Ma, Pengcheng Yang, Tianyu Liu, Peng Li, Jie Zhou, Xu Sun

Text Generation

Challenge:	Existing methods for table-to-text generation use encoder-decoder framework, but lack of large parallel data is a problem for many domains.
Approach:	They propose a model to separate table-to-text generation into two stages: key fact prediction and surface realization.
Outcome:	The proposed model achieves 27.34 BLEU score with only 1,000 parallel data, while the baseline model only achieves 9.71 BLUE score.

Do Pre-trained Models Benefit Knowledge Graph Completion? A Reliable Evaluation and a Reasonable Approach (2022.findings-acl)

Copied to clipboard

Xin Lv, Yankai Lin, Yixin Cao, Lei Hou, Juanzi Li, Zhiyuan Liu, Peng Li, Jie Zhou

Manual Evaluation Matters: Reviewing Test Protocols of Distantly Supervised Relation Extraction (2021.findings-acl)

Challenge:	Pre-trained language models capture factual knowledge from massive texts . but they are still quite behind the SOTA KGC models in terms of performance .
Approach:	They propose to use open-world assumption to evaluate PLM-based knowledge graph completion models . they propose to convert each triple and its support information into natural prompt sentences .
Outcome:	The proposed model is more accurate under the open-world assumption (OWA) this setting manual checks the correctness of knowledge that is not in KGs.

Copied to clipboard

Tianyu Gao, Xu Han, Yuzhuo Bai, Keyue Qiu, Zhiyu Xie, Yankai Lin, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou

Learning to Recover from Multi-Modality Errors for Non-Autoregressive Neural Machine Translation (2020.acl-main)

Challenge:	Distantly supervised relation extraction (RE) has attracted much attention in the past few years . previous methods to evaluate models manually or directly on autolabeled data have produced inaccurate evaluations .
Approach:	They propose to use distant supervision to generate large-scale autolabeled data . they build manually-annotated test sets for two DS-RE datasets and evaluate models .
Outcome:	The proposed method produces 53% wrong labels at the entity pair level in the popular NYT10 dataset.

Copied to clipboard

Qiu Ran, Yankai Lin, Peng Li, Jie Zhou

Multilinguality

Challenge:	Existing non-autoregressive neural machine translation models suffer from multi-modality problem . despite their autoregressivity, most NMT models suffer with slow decoding speed .
Approach:	They propose a semi-autoregressive model which generates a translation as a sequence of segments while each segment is predicted token-by-token.
Outcome:	The proposed model can achieve 4 times speedup while maintaining comparable performance.

Context Tracking Network: Graph-based Context Modeling for Implicit Discourse Relation Recognition (2021.naacl-main)

Copied to clipboard

Yingxue Zhang, Fandong Meng, Peng Li, Ping Jian, Jie Zhou

GroupRank: A Groupwise Paradigm for Effective and Efficient Passage Reranking with LLMs (2026.findings-acl)

Challenge:	Existing models fail to fully utilize contextual information which plays an important role in interpreting sentences.
Approach:	They propose a graph-based Context Tracking Network to model the discourse context for IDRR.
Outcome:	The proposed model can integrate sentence-level and token-level contextual semantics better than existing models.

Copied to clipboard

Meixiu Long, Duolin Sun, Dan Yang, Yihan Jiao, Lei Liu, Jiahai Wang, Binbin Hu, Yue Shen, Jie Feng, Zhehao Tan, Junjie Wang, Lianzhen Zhong, Jian Wang, Peng Wei, Jinjie Gu

Learning to Win Lottery Tickets in BERT Transfer via Task-agnostic Mask Training (2022.naacl-main)

Challenge:	Existing rankers excel in lexical-matching scenarios, while they struggle with complex queries requiring deep reasoning.
Approach:	They propose a new paradigm that balances flexibility and context awareness to unlock the full potential of groupwise reranking.
Outcome:	The proposed approach achieves a state-of-the-art 65.2 NDCG@10 on BRIGHT and surpasses baselines by 2.1 points on R2MED while delivering a 6.4 inference speedup.

Copied to clipboard

Yuanxin Liu, Fandong Meng, Zheng Lin, Peng Fu, Yanan Cao, Weiping Wang, Jie Zhou

Rethinking Stealthiness of Backdoor Attack against NLP Models (2021.acl-long)

Challenge:	Recent studies show pre-trained language models contain matching subnetworks that have similar transfer learning performance as the original PLM.
Approach:	They propose to prune matching subnetworks using magnitude-based pruning . they propose to optimize the subnetwork structure towards the pre-training objectives .
Outcome:	The proposed method is more efficient in searching subnetworks and advantageous when fine-tuning within a range of data scarcity.

Copied to clipboard

Wenkai Yang, Yankai Lin, Peng Li, Jie Zhou, Xu Sun

Robustness in NLP

Challenge:	Existing backdoor attacks are not stealthy to system deployers or users.
Approach:	They propose a novel backdoor attack method based on negative data augmentation and modifying word embeddings that is much stealthier while maintaining pretty good attacking performance.
Outcome:	The proposed method is much stealthier while maintaining pretty good attacking performance.

Evolving Agentic Workflow Driven by Human-Agent Collaboration (2026.findings-acl)

Copied to clipboard

Yuxin Liu, Jinxuan Zhang, Yuezhang Peng, Hefeng Zhou, Xiangfeng Wang, Jiong Lou, Chentao Wu, Jie LI, Jingjing Qu, Chaochao Lu

Challenge:	Existing approaches to generate agentic workflows using large language models are limited by high manual design costs, inefficient agentic search, and poor dynamic adaptability to new tasks and human preferences.
Approach:	They propose an evolutionary framework for generating agentic workflows through human-agent collaboration using evolutionary algorithms that mutate and cross over their structures, prompts, and LLM backbones.
Outcome:	The proposed framework surpasses other automated baselines by 27.34% while achieving comparable performance to o1-preview at only one-fourth of the cost.

Fully Hyperbolic Neural Networks (2022.acl-long)

Copied to clipboard

Weize Chen, Xu Han, Yankai Lin, Hexu Zhao, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou

Challenge:	Existing hyperbolic neural networks encode features in the hyperbolical space yet formalize most of their operations in the tangent space.
Approach:	They propose a fully hyperbolic framework to build hyperbolical networks based on the Lorentz model by adapting Lorentzer transformations to formalize essential operations of neural networks.
Outcome:	The proposed framework has better performance on four NLP tasks compared with existing hyperbolic models .

Question-Interlocutor Scope Realized Graph Modeling over Key Utterances for Dialogue Reading Comprehension (2023.findings-acl)

Copied to clipboard

Jiangnan Li, Mo Yu, Fandong Meng, Zheng Lin, Peng Fu, Weiping Wang, Jie Zhou

Natural Language Interfaces

Challenge:	Compared to standard RC tasks, dialogue reading comprehension (DRC) has raised challenges because of the complex speaker information and noisy dialogue context.
Approach:	They propose a new method for dialogue reading comprehension that extracts answers from dialogues by using key-utterances-extracting methods and a Question-Interlocutor Scope Realized Graph.
Outcome:	The proposed method achieves state-of-the-art performance against previous works.

MirrorQA: Benchmarking Multimodal LLMs on Mirror-Orientation Reasoning (2026.acl-long)

Copied to clipboard

Jingping Liu, Xingchen Peng, Yan Zhou, Ziyan Liu, Jie Zhai, Ronghao Chen, Huacan Wang, Xiaofeng Jia

HMEAE: Hierarchical Modular Event Argument Extraction (D19-1)

Challenge:	Multimodal large language models (MLLMs) have achieved remarkable progress in recent years, yet their ability to perform left–right reasoning in mirror contexts remains underexplored.
Approach:	They propose a benchmark to evaluate MLLMs' ability to distinguish left from right from a subject-centered perspective.
Outcome:	The proposed benchmarks show that even the best performing models achieve only 65.40% accuracy, far below the 99.28% accuracy of humans.

Copied to clipboard

Xiaozhi Wang, Ziqi Wang, Xu Han, Zhiyuan Liu, Juanzi Li, Peng Li, Maosong Sun, Jie Zhou, Xiang Ren

XDailyDialog: A Multilingual Parallel Dialogue Corpus (2023.acl-long)

Challenge:	Existing event extraction methods classify each argument role independently, ignoring conceptual correlations between different argument roles.
Approach:	They propose a Hierarchical Modular Event Argument Extraction model to provide inductive bias from the concept hierarchy of event argument roles.
Outcome:	The proposed model outperforms existing methods on real-world datasets and shows that it leverages useful knowledge from the concept hierarchy.

Copied to clipboard

Zeming Liu, Ping Nie, Jie Cai, Haifeng Wang, Zheng-Yu Niu, Peng Zhang, Mrinmaya Sachan, Kaiping Peng

Natural Language Interfaces

Challenge:	Existing datasets for open-domain dialogue modeling limited to a single language . absence of multilingual datasets hinders development of robust open- domain dialog systems .
Approach:	They propose a multilingual parallel open-domain dialog dataset to explore multilingual and cross-lingual open- domain dialog.
Outcome:	The proposed model can be used to explore multilingual and cross-lingual open-domain dialogs in other languages.

Multimodal Incremental Transformer with Visual Grounding for Visual Dialogue Generation (2021.findings-acl)

Copied to clipboard

Feilong Chen, Fandong Meng, Xiuyi Chen, Peng Li, Jie Zhou

Dialogue Systems & Conversational Agents

Challenge:	Existing studies focus on implicit exploration of multimodal coreference but neglect the importance of locating the objects explicitly in the visual content, which is associated with textual entities.
Approach:	They propose a multimodal incremental transformer with visual grounding which aims to explicitly locate related objects in the image guided by textual entities.
Outcome:	The proposed model achieves comparable performance on the VisDial v0.9 and v1.0 datasets.

ROSE: Robust Selective Fine-tuning for Pre-trained Language Models (2022.emnlp-main)

Copied to clipboard

Lan Jiang, Hao Zhou, Yankai Lin, Peng Li, Jie Zhou, Rui Jiang

Robustness in NLP

Challenge:	Recent studies have highlighted the lack of adversarial robustness in pre-trained models.
Approach:	They propose a fine-tuning approach that conducts selective updates when adapting pre-trained models to downstream tasks.
Outcome:	The proposed approach improves adversarial robustness on downstream tasks . it eliminates spurious updates, leading to flatter and wider optima than the conventional method .

Search-P1: Path-Centric Reward Shaping for Stable and Efficient Agentic RAG Training (2026.acl-industry)

Copied to clipboard

Tianle Xia, Ming Xu, Lingxiang Hu, Yiding Sun, Wenwei Li, Linfang Shang, Liqun Liu, Peng Shu, Huan Yu, Jie Jiang

RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language Models (2024.findings-acl)

Challenge:	Retrieval-Augmented Generation (RAG) enhances large language models by incorporating external knowledge, yet traditional singleround retrieval struggles with complex multistep reasoning.
Approach:	They propose a framework that introduces path-centric reward shaping for agentic RAG training.
Outcome:	The proposed framework improves on existing methods with an average accuracy gain of 7.7 points.

Copied to clipboard

Noah Wang, Z.y. Peng, Haoran Que, Jiaheng Liu, Wangchunshu Zhou, Yuhan Wu, Hongcheng Guo, Ruitong Gan, Zehao Ni, Jian Yang, Man Zhang, Zhaoxiang Zhang, Wanli Ouyang, Ke Xu, Wenhao Huang, Jie Fu, Junran Peng

From Mimicking to Integrating: Knowledge Integration for Pre-Trained Language Models (2022.findings-emnlp)

Challenge:	Large Language Models (LLMs) have paved the way for complex tasks such as role-playing.
Approach:	They propose a framework to benchmark, elicit, and enhance role-playing abilities in Large Language Models.
Outcome:	The proposed framework improves role-playing abilities with 168,093 samples.

Copied to clipboard

Lei Li, Yankai Lin, Xuancheng Ren, Guangxiang Zhao, Peng Li, Jie Zhou, Xu Sun

ERICA: Improving Entity and Relation Understanding for Pre-trained Language Models via Contrastive Learning (2021.acl-long)

Challenge:	Existing models for natural language processing (NLP) are fine-tuned and released for research and deployments.
Approach:	They propose a PLM reuse paradigm that merges teacher-PLM knowledge into a student model.
Outcome:	The proposed paradigm can reduce the computational cost and environmental side-effects of retraining the PLM from scratch.

Copied to clipboard

Yujia Qin, Yankai Lin, Ryuichi Takanobu, Zhiyuan Liu, Peng Li, Heng Ji, Minlie Huang, Maosong Sun, Jie Zhou

CodRED: A Cross-Document Relation Extraction Dataset for Acquiring Knowledge in the Wild (2021.emnlp-main)

Challenge:	Existing pre-training objectives do not explicitly model relational facts in text . Experimental results show that ERICA can improve typical PLMs on several language understanding tasks, including relation extraction, entity typing and question answering.
Approach:	They propose a contrastive learning framework ERICA to obtain a deep understanding of entities and relations in text.
Outcome:	The proposed framework can improve PLMs on several language understanding tasks, especially under low-resource settings.

Copied to clipboard

Yuan Yao, Jiaju Du, Yankai Lin, Peng Li, Zhiyuan Liu, Jie Zhou, Maosong Sun

TGEA: An Error-Annotated Dataset and Benchmark Tasks for TextGeneration from Pretrained Language Models (2021.acl-long)

Challenge:	Existing relation extraction methods focus on extracting relational facts between entity pairs within single sentences or documents.
Approach:	They present a problem of cross-document relation extraction (CRE) using human annotations.
Outcome:	The proposed dataset is the first human-annotated cross-document RE dataset . it shows that it is challenging to existing RE methods including strong BERT-based models.

Copied to clipboard

Jie He, Bo Peng, Yi Liao, Qun Liu, Deyi Xiong

T2R-BENCH: A Benchmark for Real World Table-to-Report Task (2025.emnlp-main)

Challenge:	Using pretrained language models, we propose an error-annotated dataset for text generation . we use carefully selected prompt words to guide GPT-2 to generate candidate sentences .
Approach:	They propose an error-annotated dataset with multiple benchmark tasks for text generation from pretrained language models.
Outcome:	The proposed dataset covers 24 types of errors according to common sense and linguistics.

Copied to clipboard

Jie Zhang, Changzai Pan, Sishi Xiong, Kaiwen Wei, Yu Zhao, Xiangyu Li, Jiaxin Peng, Xiaoyan Gu, Jian Yang, Wenhan Chang, Zhenhe Wu, Jiang Zhong, Shuangyong Song, Xuelong Li

GKD: A General Knowledge Distillation Framework for Large-scale Pre-trained Language Model (2023.acl-industry)

Challenge:	Existing table benchmarks lack the capacity to adequately assess the practical application of table reasoning in industrial applications.
Approach:	They propose a bilingual table-to-report task and a table-based benchmark to assess the quality of table reasoning.
Outcome:	The proposed task is based on a bilingual benchmark with 457 industrial tables and evaluation criteria to measure the quality of report generation.

Copied to clipboard

Shicheng Tan, Weng Lam Tam, Yuanchun Wang, Wenwen Gong, Shu Zhao, Peng Zhang, Jie Tang

CharacterGLM: Customizing Social Characters with Large Language Models (2024.emnlp-industry)

Challenge:	Existing knowledge distillation frameworks for language models are limited by memory and the use of complex distillation methods on larger-scale PLMs.
Approach:	They propose a general knowledge distillation framework that supports distillation on larger-scale PLMs using various distillation methods.
Outcome:	The proposed framework can support distillation on larger-scale PLMs and 25 mainstream methods on 8 NVIDIA A100 (40GB) GPUs.

Copied to clipboard

Jinfeng Zhou, Zhuang Chen, Dazhen Wan, Bosi Wen, Yi Song, Jifan Yu, Yongkang Huang, Pei Ke, Guanqun Bi, Libiao Peng, JiaMing Yang, Xiyao Xiao, Sahand Sabour, Xiaohan Zhang, Wenjing Hou, Yijia Zhang, Yuxiao Dong, Hongning Wang, Jie Tang, Minlie Huang

LLMBox: A Comprehensive Library for Large Language Models (2024.acl-demos)

Challenge:	Character-based dialogue systems (CharacterDial) allow users to customize social characters for social interactions.
Approach:	They will collect a large-scale Chinese corpus of characters with diverse categories and behaviors and develop CharacterGLM models to address these challenges.
Outcome:	Experiments show that CharacterGLM outperforms most popular open- and closed-source LLMs and performs comparable to GPT-4.

Copied to clipboard

Tianyi Tang, Hu Yiwen, Bingqian Li, Wenyang Luo, ZiJing Qin, Haoxiang Sun, Jiapeng Wang, Shiyi Xu, Xiaoxue Cheng, Geyang Guo, Han Peng, Bowen Zheng, Yiru Tang, Yingqian Min, Yushuo Chen, Jie Chen, Ranchi Zhao, Luran Ding, Yuhao Wang, Zican Dong, Xia Chunxuan, Junyi Li, Kun Zhou, Xin Zhao, Ji-Rong Wen

Advancing MoE Efficiency: A Collaboration-Constrained Routing (C2R) Strategy for Better Expert Parallelism Design (2025.naacl-long)

Challenge:	a library to facilitate the development, use, and evaluation of large language models (LLMs) is presented.
Approach:	They propose a unified library to facilitate the development, use and evaluation of large language models (LLMs).
Outcome:	The proposed library is based on extensive experiments in a variety of evaluation settings.

Copied to clipboard

Mohan Zhang, Pingzhi Li, Jie Peng, Mufan Qiu, Tianlong Chen

Green & Sustainable NLP

Challenge:	Using Mixture-of-Experts, researchers have found that efficient MoE is difficult to achieve due to two key reasons: imbalanced expert activation and massive communication overhead.
Approach:	They propose a collaboration-constrained routing strategy that encourages more specialized expert groups and leverages expert specialization.
Outcome:	The proposed approach achieves an average performance improvement of 0.51% and 0.33% on LLaMA-MoE and Qwen-MaE respectively.

Continual Relation Learning via Episodic Memory Activation and Reconsolidation (2020.acl-main)

Copied to clipboard

Xu Han, Yi Dai, Tianyu Gao, Yankai Lin, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou

Challenge:	Existing methods to learn incessantly emerging novel relations are overfitting the few memorized examples of old relations, causing confusion among existing relations.
Approach:	They introduce episodic memory activation and reconsolidation (EMAR) to continual relation learning.
Outcome:	The proposed method outperforms state-of-the-art models in catastrophic forgetting old relations.

Dialect-SQL: An Adaptive Framework for Bridging the Dialect Gap in Text-to-SQL (2025.emnlp-main)

Copied to clipboard

Jie Shi, Xi Cao, Bo Xu, Jiaqing Liang, Yanghua Xiao, Jia Chen, Peng Wang, Wei Wang

Programming Languages in NLP

Challenge:	Existing Text-to-SQL research focuses on specific database systems, limiting adaptability to different dialects.
Approach:	They propose a framework that employs Object Relational Mapping (ORM) code as an intermediate language to bridge this gap.
Outcome:	The proposed framework outperforms existing methods that generate SQL queries directly.

GoG: Relation-aware Graph-over-Graph Network for Visual Dialog (2021.findings-acl)

Copied to clipboard

Feilong Chen, Xiuyi Chen, Fandong Meng, Peng Li, Jie Zhou

Bridging the Gap between Prior and Posterior Knowledge Selection for Knowledge-Grounded Dialogue Generation (2020.emnlp-main)

Challenge:	Experimental results show that our model outperforms the strong baseline in both generative and discriminative settings by a significant margin.
Approach:	They propose a relation-aware graph-over-graph network (GoG) for visual dialog . their model outperforms the strong baseline in both generative and discriminative settings .
Outcome:	The proposed model outperforms baseline models in both generative and discriminative settings by a significant margin.

Copied to clipboard

Xiuyi Chen, Fandong Meng, Peng Li, Feilong Chen, Shuang Xu, Bo Xu, Jie Zhou

Dialogue Systems & Conversational Agents

Challenge:	Existing knowledge-grounded dialogue models lack prior and posterior knowledge selection . prior selection module may not learn to select knowledge properly because of lack of posterior information .
Approach:	They propose a knowledge distillation-based training strategy to remove the exposure bias of knowledge selection.
Outcome:	The proposed model improves on two knowledge-grounded dialogue datasets.

ELLE: Efficient Lifelong Pre-training for Emerging Data (2022.findings-acl)

Copied to clipboard

Yujia Qin, Jiajie Zhang, Yankai Lin, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou

SSR-A: Spatial- and Semantic-Aware Instructions and Curriculum Reinforcement for Advertisement Compliant Rectification (2026.acl-industry)

Challenge:	Existing pre-trained language models are typically trained with static data, ignoring that streaming data of various sources may continuously grow.
Approach:	They propose to use function preserved model expansion to expand existing PLM's width and depth to improve efficiency of knowledge acquisition.
Outcome:	The proposed model improves pre-training efficiency and performance over existing models on BERT and GPT.

Copied to clipboard

Cao Te, Mengge Xue, Zhenyu Hu, Yuan Chen, Liqun Liu, Peng Shu, Huan Yu, Jie Jiang

Learning from Context or Names? An Empirical Study on Neural Relation Extraction (2020.emnlp-main)

Challenge:	Existing methods to fix non-compliant images suffer from over-editing, destroying original intent and perceptual similarity.
Approach:	They propose a framework for the minimalist rectification of non-compliant image ads.
Outcome:	The proposed framework outperforms state-of-the-art baselines in both compliance and preservation of visual and commercial consistency.

Copied to clipboard

Hao Peng, Tianyu Gao, Xu Han, Yankai Lin, Peng Li, Zhiyuan Liu, Maosong Sun, Jie Zhou

Language Prior Is Not the Only Shortcut: A Benchmark for Shortcut Learning in VQA (2022.findings-emnlp)

Challenge:	Existing datasets may leak shallow heuristics via entity mentions, thus contributing to the high performance on RE benchmarks.
Approach:	They propose an entity-masked contrastive framework for relation extraction to gain a deeper understanding on textual context and type information while avoiding rote memorization of entities.
Outcome:	The proposed framework improves the effectiveness and robustness of neural models in different RE scenarios.

Copied to clipboard

Qingyi Si, Fandong Meng, Mingyu Zheng, Zheng Lin, Yuanxin Liu, Peng Fu, Yanan Cao, Weiping Wang, Jie Zhou

Question Answering

Challenge:	Visual Question Answering (VQA) models are prone to learn the shortcut solution formed by dataset biases rather than the intended solution.
Approach:	They propose a dataset that considers varying types of shortcuts by constructing different distribution shifts in multiple OOD test sets.
Outcome:	The proposed dataset considers varying types of shortcuts by constructing different distribution shifts in multiple OOD test sets.

ARGUS: Policy-Adaptive Ad Governance via Evolving Reinforcement with Adversarial Umpiring (2026.acl-industry)

Copied to clipboard

Deyi Ji, Junyu Lu, Xuanyi Liu, Liqun Liu, Hailong Zhang, Peng Shu, Huan Yu, Jie Jiang, Tianrun Chen, Lanyun Zhu

MulVul: Retrieval-augmented Multi-Agent Code Vulnerability Detection via Cross-Model Prompt Evolution (2026.acl-long)

Challenge:	Existing regulatory policies create label inconsistencies and reasoning ambiguities in historical datasets.
Approach:	They propose a policy-adaptive governance system that enables evolving reinforcement through multi-agent adversarial umpiring.
Outcome:	The proposed system outperforms fine-tuning baselines on industrial and public datasets . it enables evolving reinforcement through multi-agent adversarial umpiring .

Copied to clipboard

Zihan Wu, Jie Xu, Yun Peng, Chun Yong Chong, Xiaohua Jia

Gen-SQL: Efficient Text-to-SQL By Bridging Natural Language Question And Database Schema With Pseudo-Schema (2025.coling-main)

Challenge:	Large Language Models (LLMs) struggle to automate real-world vulnerability detection due to the heterogeneity of vulnerability patterns and manual prompt engineering for massive weakness categories is unscalable.
Approach:	They propose a retrieval-augmented multi-agent framework for precise and broad-coverage vulnerability detection using a coarse-to-fine strategy.
Outcome:	The proposed framework outperforms the baseline model on 130 CWE types and achieves 34.79% Macro-F1 performance.

Copied to clipboard

Jie Shi, Bo Xu, Jiaqing Liang, Yanghua Xiao, Jia Chen, Chenhao Xie, Peng Wang, Wei Wang

Programming Languages in NLP

Challenge:	Recent studies have shifted paradigms and leveraged Large Language Models (LLMs) to tackle the challenging task of Text-to-SQL.
Approach:	They propose a framework that leverages large language models to generate SQL queries . they exploit prior knowledge from the LLM to enhance embedding-based retriever .
Outcome:	The proposed method improves embedding-based retriever and reduces cost.

Dynamic Knowledge Distillation for Pre-trained Language Models (2021.emnlp-main)

Copied to clipboard

Lei Li, Yankai Lin, Shuhuai Ren, Peng Li, Jie Zhou, Xu Sun

Glider: Global and Local Instruction-Driven Expert Router (2025.emnlp-main)

Challenge:	Existing methods conduct knowledge distillation statically, e.g., student model aligns output distribution to teacher model on pre-defined training dataset.
Approach:	They propose a dynamic knowledge distillation that empowers the student to adjust the learning procedure according to its competency . they find it is promising and provide discussions on potential future directions towards more efficient methods .
Outcome:	The proposed method can boost student model performance while accelerating training . the proposed method reduces memory usage and accelerates model inference .

Copied to clipboard

Pingzhi Li, Prateek Yadav, Jaehong Yoon, Jie Peng, Yi-Lin Sung, Mohit Bansal, Tianlong Chen

CascadeBERT: Accelerating Inference of Pre-trained Language Models via Calibrated Complete Models Cascade (2021.findings-emnlp)

Challenge:	Existing methods for routing-based expert models favor generalization over performance on held-in tasks.
Approach:	They propose a global and local instruction driven expert router that leverages recent LLMs' semantic reasoning capabilities to generate task-specific instructions from the input query.
Outcome:	The proposed method improves held-in performance while maintaining strong generalization on held-out tasks.

Copied to clipboard

Lei Li, Yankai Lin, Deli Chen, Shuhuai Ren, Peng Li, Jie Zhou, Xu Sun

DocRED: A Large-Scale Document-Level Relation Extraction Dataset (P19-1)

Challenge:	Experimental results show that CascadeBERT can achieve an overall 15% improvement under 4x speed-up compared with existing dynamic early exiting methods on six classification tasks.
Approach:	They propose a framework which emits predictions in internal layers without passing through the entire model.
Outcome:	The proposed framework can achieve 15% improvement under 4x speed-up compared with existing methods on six classification tasks yielding more calibrated and accurate predictions.

Copied to clipboard

Yuan Yao, Deming Ye, Peng Li, Xu Han, Yankai Lin, Zhenghao Liu, Zhiyuan Liu, Lixin Huang, Jie Zhou, Maosong Sun