Papers by Zhiyuan Xu

Fine-mixing: Mitigating Backdoors in Fine-tuned Language Models (2022.findings-emnlp)

Challenge:	Existing methods for learning word and entity representations in monolingual settings are limited.
Approach:	They propose a method for joint representation learning of cross-lingual words and entities that captures mutually complementary knowledge and enables cross-linguistic inferences.
Outcome:	The proposed method captures mutually complementary knowledge and enables cross-lingual inferences among knowledge bases and texts.

Copied to clipboard

Zhiyuan Zhang, Lingjuan Lyu, Xingjun Ma, Chenguang Wang, Xu Sun

STARS: A Unified Framework for Singing Transcription, Alignment, and Refined Style Annotation (2025.findings-acl)

Challenge:	Existing methods for defending NLP models against backdoors have ignored the clean weights of PLMs.
Approach:	They exploit pre-trained weights to mitigate backdoors in fine-tuned NLP models . they use a fine-mixing technique and an Embedding Purification technique to do the same .
Outcome:	The proposed method outperforms baseline mitigation methods on three single-sentence sentiment classification tasks and two sentence-pair classification tasks.

Copied to clipboard

Wenxiang Guo, Yu Zhang, Changhao Pan, Zhiyuan Zhu, Ruiqi Li, ZheTao Chen, Wenhao Xu, Fei Wu, Zhou Zhao

Speech & Audio in NLP

Challenge:	Existing automated singing annotation (ASA) methods tackle isolated aspects of the annotation pipeline.
Approach:	They propose a framework that addresses transcription, alignment, and refined style annotations.
Outcome:	The proposed framework delivers comprehensive multi-level annotations encompassing: (1) precise phoneme-audio alignment, (2) robust note transcription and temporal localization, (3) expressive vocal technique identification, and (4) global stylistic characterization including emotion and pace.

RAGEval: Scenario Specific RAG Evaluation Dataset Generation Framework (2025.acl-long)

Copied to clipboard

Kunlun Zhu, Yifan Luo, Dingling Xu, Yukun Yan, Zhenghao Liu, Shi Yu, Ruobing Wang, Shuo Wang, Yishan Li, Nan Zhang, Xu Han, Zhiyuan Liu, Maosong Sun

Text Generation

Challenge:	Existing evaluation metrics for RAG systems are lacking due to high costs of data construction and lack of factual accuracy.
Approach:	They propose a framework to evaluate RAG systems in specialized scenarios . they propose three new metrics to evaluate LLM-generated responses .
Outcome:	The proposed framework outperforms zero-shot and one-shot methods in terms of clarity, safety, conformity, and richness of generated samples.

Adapting Meta Knowledge Graph Information for Multi-Hop Reasoning over Few-Shot Relations (D19-1)

Copied to clipboard

Xin Lv, Yuxian Gu, Xu Han, Lei Hou, Juanzi Li, Zhiyuan Liu

TritonBench: Benchmarking Large Language Model Capabilities for Generating Triton Operators (2025.findings-acl)

Challenge:	Existing methods for multi-hop reasoning assume that every relation has enough triples for training . however, performance drops significantly on few-shot relations .
Approach:	They propose a meta-based multi-hop reasoning method that learns meta parameters from high-frequency relations that could quickly adapt to few-shot scenarios.
Outcome:	The proposed method outperforms state-of-the-art methods in few-shot scenarios on two public datasets from Freebase and NELL.

Copied to clipboard

Jianling Li, ShangZhan Li, Zhenye Gao, Qi Shi, Yuxuan Li, Zefan Wang, Jiacheng Huang, WangHaojie WangHaojie, Jianrong Wang, Xu Han, Zhiyuan Liu, Maosong Sun

Disentangled Knowledge Transfer for OOD Intent Discovery with Unified Contrastive Learning (2022.acl-short)

Challenge:	Triton is a high-level Python-like programming language for building efficient GPU kernels.
Approach:	They propose a TritonBench benchmark that provides a comprehensive evaluation of Tritonic operators on widely deployed GPUs.
Outcome:	The proposed benchmarks show that current LLMs struggle to generate efficient Triton operators on widely deployed GPUs aligned with industry applications.

Copied to clipboard

Yutao Mou, Keqing He, Yanan Wu, Zhiyuan Zeng, Hong Xu, Huixing Jiang, Wei Wu, Weiran Xu

Gentopia.AI: A Collaborative Platform for Tool-Augmented LLMs (2023.emnlp-demo)

Challenge:	Existing methods to find out out-of-domain (OOD) intents do not take prior knowledge of in-domain data into account.
Approach:	They propose a disentangled knowledge transfer method to bridge the gap between IND pre-training and OOD clustering by using a unified multi-head contrastive learning framework.
Outcome:	The proposed method is able to group new unknown intents into different clusters, enabling future development of the system.

Copied to clipboard

Binfeng Xu, Xukun Liu, Hua Shen, Zeyu Han, Yuhan Li, Murong Yue, Zhiyuan Peng, Yuchen Liu, Ziyu Yao, Dongkuan Xu

Diffusion Theory as a Scalpel: Detecting and Purifying Poisonous Dimensions in Pre-trained Language Models Caused by Backdoor or Bias (2023.findings-acl)

Challenge:	Existing frameworks for Augmented Language Models lack flexibility, democratization, and holistic evaluation.
Approach:	They propose a lightweight and extensible framework for Augmented Language Models called Gentopia.
Outcome:	The proposed framework integrates language models, task formats, prompting modules, and plugins into a unified paradigm.

Copied to clipboard

Zhiyuan Zhang, Deli Chen, Hao Zhou, Fandong Meng, Jie Zhou, Xu Sun

Exploring Mode Connectivity for Pre-trained Language Models (2022.emnlp-main)

Challenge:	Existing methods to fine-tune pre-trained language models (PLMs) are not safe, since the fine-uning process is invisible to the user.
Approach:	They propose a technique to study the dynamic process of fine-tuning for finding poisonous dimensions using diffusion theory.
Outcome:	The proposed approach can detect poisonous dimensions with abnormal dynamics, purify them and fine-tune them on a clean dataset.

Copied to clipboard

Yujia Qin, Cheng Qian, Jing Yi, Weize Chen, Yankai Lin, Xu Han, Zhiyuan Liu, Maosong Sun, Jie Zhou

Knowledge Inheritance for Pre-trained Language Models (2022.naacl-main)

Challenge:	Recent years have witnessed the prevalent application of pre-trained language models (PLMs) in NLP. From the perspective of parameter space, PLMs provide generic initialization, starting from which high-performance minima could be found.
Approach:	They investigate the geometric connections of different minima through the lens of mode connectivity, which measures whether two minima can be connected with a low-loss path.
Outcome:	The proposed model can be used to find low-loss paths between two minima, and to understand how their mode connectivity affects their task knowledge.

Copied to clipboard

Yujia Qin, Yankai Lin, Jing Yi, Jiajie Zhang, Xu Han, Zhengyan Zhang, Yusheng Su, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou

MAVEN: A Massive General Domain Event Detection Dataset (2020.emnlp-main)

Challenge:	Existing large-scale pre-trained language models are mainly trained from scratch individually, ignoring that many well-taught PLMs are available.
Approach:	They propose a pre-training framework called knowledge inheritance and propose auxiliary supervision to efficiently learn larger PLMs.
Outcome:	The proposed framework can be used to train large-scale language models with huge parameters and a large dataset can be adapted to domain adaptation and knowledge transfer.

Copied to clipboard

Xiaozhi Wang, Ziqi Wang, Xu Han, Wangyi Jiang, Rong Han, Zhiyuan Liu, Juanzi Li, Peng Li, Yankai Lin, Jie Zhou

MAVEN-ARG: Completing the Puzzle of All-in-One Event Understanding Dataset with Event Argument Annotation (2024.acl-long)

Challenge:	Existing datasets exhibit data scarcity and limited coverage of general-domain events.
Approach:	They present a MAssive eVENt detection dataset which contains 4,480 Wikipedia documents and 168 event types.
Outcome:	The proposed dataset shows that existing methods cannot achieve promising results on the small datasets.

Copied to clipboard

Xiaozhi Wang, Hao Peng, Yong Guan, Kaisheng Zeng, Jianhui Chen, Lei Hou, Xu Han, Yankai Lin, Zhiyuan Liu, Ruobing Xie, Jie Zhou, Juanzi Li

Raise a Child in Large Language Model: Towards Effective and Generalizable Fine-tuning (2021.emnlp-main)

Challenge:	Existing datasets for event understanding have limited coverage due to complexity of tasks.
Approach:	They propose a dataset that augments MAVEN datasets with event argument annotations . they propose 98,591 events and 290,613 arguments obtained with laborious human annotation .
Outcome:	The proposed dataset is the first all-in-one dataset supporting event detection, event argument extraction, and event relation extraction.

Copied to clipboard

Runxin Xu, Fuli Luo, Zhiyuan Zhang, Chuanqi Tan, Baobao Chang, Songfang Huang, Fei Huang

Seq1F1B: Efficient Sequence-Level Pipeline Parallelism for Large Language Model Training (2025.naacl-long)

Challenge:	Recent pretrained language models extend from millions to billions of parameters.
Approach:	They propose a technique which forwards on a whole network while backwarding on resetting the gradients of the non-child network during the backward process.
Outcome:	The proposed technique outperforms the vanilla fine-tuning technique on various downstream tasks and can achieve better generalization performance by large margins.

Copied to clipboard

Sun Ao, Weilin Zhao, Xu Han, Cheng Yang, Xinrong Zhang, Zhiyuan Liu, Chuan Shi, Maosong Sun

LoRA-Flow: Dynamic LoRA Fusion for Large Language Models in Generative Tasks (2024.acl-long)

Challenge:	Current PP methods face severe bottlenecks, including pipeline bubbles and memory footprint.
Approach:	They propose a sequence-level one-forward-one-backward (1F1B) PP method for training LLMs on long sequences with high throughput and memory efficiency.
Outcome:	The proposed method achieves 1.14X training throughput with half memory footprint compared to baseline methods . it trains an LLM with 30B parameters on sequences up to 64k tokens using 64X NVIDIA A100 GPUs .

Copied to clipboard

Hanqing Wang, Bowen Ping, Shuo Wang, Xu Han, Yun Chen, Zhiyuan Liu, Maosong Sun

More Data, More Relations, More Context and More Openness: A Review and Outlook for Relation Extraction (2020.aacl-main)

Challenge:	LoRA-Flow uses lightweight modules to customize large language models for downstream tasks . previous work on LoRA combination relied on task-level weights for each involved LoRA .
Approach:	They propose a LoRA-Flow approach that uses dynamic weights to adjust the impact of different LoRAs.
Outcome:	The proposed method outperforms baselines with task-level weights on six generative tasks.

Copied to clipboard

Xu Han, Tianyu Gao, Yankai Lin, Hao Peng, Yaoliang Yang, Chaojun Xiao, Zhiyuan Liu, Peng Li, Jie Zhou, Maosong Sun

MinerU2.5: A Decoupled Vision-Language Model for Efficient High-Resolution Document Parsing (2026.acl-industry)

Challenge:	Existing methods for extracting relational facts from text have been successful . but with explosion of Web text, human knowledge is increasing drastically .
Approach:	They propose to improve relation extraction methods to extract relational facts from text . they analyze existing methods and show promising directions towards more powerful RE .
Outcome:	The proposed methods can extract relational facts from text, but they are still lacking in the current field.

Copied to clipboard

Junbo Niu, Zheng Liu, Zhuangcheng Gu, Bin Wang, Linke Ouyang, Zhiyuan Zhao, Tao Chu, Tianyao He, Fan Wu, Qintong Zhang, Zhenjiang Jin, Guang Liang, Rui Zhang, Wenzheng Zhang, Yuan Qu, Zhifei Ren, Yuefeng Sun, Zirui Tang, Boyu Niu, Yuanhong Zheng, Dongsheng Ma, Ziyang Miao, Hejun Dong, Siyi Qian, Junyuan Zhang, Fangdong Wang, Jingzhou Chen, Xiaomeng Zhao, Liqun Wei, Wei Li, Shasha Wang, RuiLiang Xu, Yuanyuan Cao, Lu Chen, Qianqian Wu, Huaiyu Gu, Lindong Lu, Dechen Lin, null Shenguanlin, Xuanhe Zhou, Linfeng Zhang, Yuhang Zang, Xiaoyi Dong, Jiaqi Wang, Bo Zhang, Lei Bai, Pei Chu, Weijia Li, Jiang Wu, Lijun Wu, Zhenxiang Li, Guangyu Wang, Zhongying Tu, Chao Xu, Kai Chen, Bowen Zhou, Dahua Lin, Wentao Zhang, Conghui He

Adversarial Training for Weakly Supervised Event Detection (N19-1)

Challenge:	Document images are characterized by higher resolutions, denser content, and more complex structural layouts.
Approach:	They propose a 1.2B-parameter document parsing vision-language model that decouples layout analysis from local content recognition.
Outcome:	The proposed model surpasses general-purpose and domain-specific models on multiple benchmarks while maintaining significantly lower computational overhead.

Copied to clipboard

Xiaozhi Wang, Xu Han, Zhiyuan Liu, Maosong Sun, Peng Li

KACC: A Multi-task Benchmark for Knowledge Abstraction, Concretization and Completion (2021.findings-acl)

Challenge:	Detecting and identifying events is an important subtask of event extraction.
Approach:	They build a large event-related candidate set with good coverage and apply an adversarial training mechanism to iteratively identify informative instances from the candidate set and filter out those noisy ones.
Outcome:	The proposed method significantly outperforms the state-of-the-art methods on two real-world datasets.

Copied to clipboard

Jie Zhou, Shengding Hu, Xin Lv, Cheng Yang, Zhiyuan Liu, Wei Xu, Jie Jiang, Juanzi Li, Maosong Sun

Exploring the Universal Vulnerability of Prompt-based Learning Paradigm (2022.findings-naacl)

Challenge:	Existing studies focus on partial aspects of knowledge abstraction, concretization, and completion (KACC).
Approach:	They propose a unified knowledge graph benchmark to improve existing benchmarks . they collect new datasets that contain larger concept graphs and cross-view links .
Outcome:	The proposed benchmark improves existing benchmarks in terms of dataset scale, task coverage, and difficulty.

Copied to clipboard

Lei Xu, Yangyi Chen, Ganqu Cui, Hongcheng Gao, Zhiyuan Liu

Meta-Information Guided Meta-Learning for Few-Shot Relation Classification (2020.coling-main)

Challenge:	Prompt-based learning inherits the vulnerability from pre-training, where model predictions can be misled by inserting triggers into the text.
Approach:	They propose a potential solution to mitigate this vulnerability by injecting triggers into pre-trained language models using only plain text.
Outcome:	The proposed learning paradigm inherits the vulnerability from the pre-training stage . it can totally control or severely decrease the performance of prompt-based models .

Copied to clipboard

Bowen Dong, Yuan Yao, Ruobing Xie, Tianyu Gao, Xu Han, Zhiyuan Liu, Fen Lin, Leyu Lin, Maosong Sun

HyKGE: A Hypothesis Knowledge Graph Enhanced RAG Framework for Accurate and Reliable Medical LLMs Responses (2025.acl-long)

Challenge:	Existing meta-learning models rely on implicit instance statistics and are unreliability and weak interpretability.
Approach:	They propose a meta-information guided meta-learning framework that uses semantics to guide meta- learning . experimental results demonstrate the effectiveness of the proposed framework .
Outcome:	The proposed framework can establish connections between instance-based information and semantic-based data, enabling faster initialization and adaptation.

Copied to clipboard

Xinke Jiang, Ruizhe Zhang, Yongxin Xu, Rihong Qiu, Yue Fang, Zhiyuan Wang, Jinyi Tang, Hongxin Ding, Xu Chu, Junfeng Zhao, Yasha Wang

Knowledge Representation

Challenge:	Recent approaches suffer from insufficient and repetitive knowledge retrieval, tedious and time-consuming query parsing, and monotonous knowledge utilization.
Approach:	They propose a retrieval-augmented generation framework which leverages LLMs’ powerful reasoning capacity to compensate for the incompleteness of user queries.
Outcome:	The proposed framework improves the accuracy and reliability of Large Language Models (LLMs) by combining the rich knowledge of LLMs with Hypothesis Outputs.

AgentCPM-GUI: Building Mobile-Use Agents with Reinforcement Fine-Tuning (2025.emnlp-demos)

Copied to clipboard

Zhong Zhang, Yaxi Lu, Yikun Fu, Yupeng Huo, Shenzhi Yang, Yesai Wu, Han Si, Xin Cong, Haotian Chen, Yankai Lin, Xie Xie, Wei Zhou, Wang Xu, Zhou Su, Zhongwu Zhai, Xiaoming Liu, null Meiyudong, Jianming Xu, Hongyan Tian, Chongyi Wang, Chi Chen, Yuan Yao, Zhiyuan Liu, Maosong Sun

Few-NERD: A Few-shot Named Entity Recognition Dataset (2021.acl-long)

Challenge:	Large language model agents have enabled GUI-based automation, but their deployment is limited by noisy data, poor generalization, and lack of support for non-English GUIs.
Approach:	They propose an 8B-parameter GUI agent built for robust and efficient on-device GUI interaction.
Outcome:	The proposed GUI agent achieves promising performance on five public benchmarks and proposed Chinese benchmark CAGUI.

Copied to clipboard

Ning Ding, Guangwei Xu, Yulin Chen, Xiaobin Wang, Xu Han, Pengjun Xie, Haitao Zheng, Zhiyuan Liu

Low-Resource NLP

Challenge:	Existing approaches to few-shot named entity recognition (NER) focus on coarse-grained entities with few examples, while most unseen entities are fine-grounded.
Approach:	They present a human-annotated few-shot named entity recognition dataset . they construct benchmark tasks to assess the generalization capability of models .
Outcome:	The proposed model is the first few-shot NER dataset and the largest human-crafted NER data set.

KBAlign: Efficient Self Adaptation on Specific Textual Knowledge Bases (2025.findings-emnlp)

Copied to clipboard

Zheni Zeng, Yuxuan Chen, Shi Yu, Ruobing Wang, Yukun Yan, Zhenghao Liu, Shuo Wang, Xu Han, Zhiyuan Liu, Maosong Sun

APB: Accelerating Distributed Long-Context Inference by Passing Compressed Context Blocks across GPUs (2025.acl-long)

Challenge:	Existing methods for retrieval-augmented generation (RAG) are limited and fine-tuning incurs prohibitive costs of external signals.
Approach:	They propose a self-supervised framework that enhances RAG systems through efficient model adaptation.
Outcome:	The proposed framework achieves 90% of the performance gain obtained through GPT-4-supervised adaptation while relying entirely on self-annotation of much smaller models.

Copied to clipboard

Yuxiang Huang, Mingye Li, Xu Han, Chaojun Xiao, Weilin Zhao, Sun Ao, Hao Zhou, Jie Zhou, Zhiyuan Liu, Maosong Sun

Turn the Combination Lock: Learnable Textual Backdoor Attacks via Word Substitution (2021.acl-long)

Challenge:	Long-context inference is crucial for advancing large language models, but its prefill speed remains a bottleneck.
Approach:	They propose an efficient long-context inference framework that leverages multi-host approximate attention to enhance prefill speed.
Outcome:	The proposed framework achieves speedups of 9.2, 4.2, and 1.6 without any degradation in performance.

Copied to clipboard

Fanchao Qi, Yuan Yao, Sophia Xu, Zhiyuan Liu, Maosong Sun

Robustness in NLP

Challenge:	Recent studies show that neural natural language processing models are vulnerable to backdoor attacks.
Approach:	They propose to inject neural models with backdoors activated by word substitution . their results raise a serious alarm to the security of NLP models, they argue .
Outcome:	The proposed backdoors are activated by a learnable combination of word substitution and exhibit higher invisibility than previous methods.

GEAR: Graph-based Evidence Aggregating and Reasoning for Fact Verification (P19-1)

Copied to clipboard

Jie Zhou, Xu Han, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng Li, Maosong Sun

Reasoning

Challenge:	Existing methods to extract information from evidence are unable to grasp relational and logical information among the evidence.
Approach:	They propose a graph-based evidence aggregating and reasoning framework to integrate evidence from multiple pieces of evidence.
Outcome:	The proposed framework achieves significant performance improvements on a large-scale benchmark dataset.

RepoGenesis: Benchmarking End-to-End Microservice Generation from Readme to Repository (2026.acl-long)

Copied to clipboard

Zhiyuan Peng, Xin Yin, Pu Zhao, Fangkai Yang, Lu Wang, Ran Jia, Xu Chen, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang

Challenge:	Existing benchmarks focus on isolated function/class-level generation, neglecting complete microservice repository generation.
Approach:	They propose a multilingual benchmark for repository-level end-to-end web microservice generation that reflects real-world development workflows.
Outcome:	The benchmark compared 106 repositories across 18 domains and 11 frameworks and 1,258 API endpoints and 2,335 test cases.

Large Language Model-based Human-Agent Collaboration for Complex Task Solving (2024.findings-emnlp)

Copied to clipboard

Xueyang Feng, Zhi-Yuan Chen, Yujia Qin, Yankai Lin, Xu Chen, Zhiyuan Liu, Ji-Rong Wen

Neural Gibbs Sampling for Joint Event Argument Extraction (2020.aacl-main)

Challenge:	Recent advances in large language models have led to the development of LLM-based autonomous agents.
Approach:	They propose a Reinforcement Learning-based Human-Agent Collaboration method which trains a policy model to determine the most opportune stages for human intervention within the task-solving process.
Outcome:	The proposed method improves human-agent collaboration significantly through well-planned, limited human intervention.

Copied to clipboard

Xiaozhi Wang, Shengyu Jia, Xu Han, Zhiyuan Liu, Juanzi Li, Peng Li, Jie Zhou

ITAKE: Interactive Unstructured Text Annotation and Knowledge Extraction System with LLMs and ModelOps (2024.acl-demos)

Challenge:	Existing methods for event argument extraction cannot adequately model the correlation between event arguments and their roles.
Approach:	They propose a Bayesian model to jointly extract event arguments using Gibbs sampling . they train two neural networks to model prior distribution and conditional distribution over event arguments .
Outcome:	The proposed model can achieve comparable results to existing methods on two widely-used datasets.

Copied to clipboard

Jiahe Song, Hongxin Ding, Zhiyuan Wang, Yongxin Xu, Yasha Wang, Junfeng Zhao

RA2FD: Distilling Faithfulness into Efficient Dialogue Systems (2024.emnlp-main)

Challenge:	Unstructured text data contains a large amount of valuable knowledge, but there are many tools that do not meet the needs of actual business.
Approach:	They propose an unstructured text annotation and knowledge extraction system that integrates Large Language Models and ModelOps to improve model supervision and performance.
Outcome:	The proposed system integrates large language models and ModelOps to improve performance in low-resource contexts.

Copied to clipboard

Zhiyuan Zhu, Yusheng Liao, Chenxin Xu, Yunfeng Guan, Yanfeng Wang, Yu Wang

OpenNRE: An Open and Extensible Toolkit for Neural Relation Extraction (D19-3)

Challenge:	Retrieval Augmented Generation (RAG) is effective but inference inefficient, while Retrieral Free Generations (RFG) are more efficient but sacrifice faithfulness.
Approach:	They propose a retrieval-free model training scheme that uses a teacher-student framework to distill the faithfulness capacity of a student's knowledge-infused responses.
Outcome:	The proposed model surpasses the previous SOTA RFG model on knowledge-grounded dialogue datasets by an average of 33% while improving inference efficiency.

Copied to clipboard

Xu Han, Tianyu Gao, Yuan Yao, Deming Ye, Zhiyuan Liu, Maosong Sun

Stochastic Bridges as Effective Regularizers for Parameter-Efficient Tuning (2023.findings-acl)

Challenge:	OpenNRE provides a framework to implement neural relation extraction (RE) . the toolkit provides various functional modules based on TensorFlow and PyTorch .
Approach:	OpenNRE is an open-source framework to implement neural relation extraction models. they also release an online system to meet real-time extraction without any training and deployment.
Outcome:	OpenNRE provides a framework to implement neural models for relation extraction (RE) the toolkit also includes an online system to meet real-time extraction without training and deployment .

Copied to clipboard

Weize Chen, Xu Han, Yankai Lin, Zhiyuan Liu, Maosong Sun, Jie Zhou

Modeling Discriminative Representations for Out-of-Domain Detection with Supervised Contrastive Learning (2021.acl-short)

Challenge:	Existing methods for tuning pre-trained language models ignore the running cost and only optimize the terminal cost.
Approach:	They propose to use stochastic bridges to regularize intermediate states and use regularization as running cost of PETs.
Outcome:	The proposed methods can be used to tune large pre-trained language models . they can be compared to full-parameter fine-tuning by tuning a small number of parameters .

Copied to clipboard

Zhiyuan Zeng, Keqing He, Yuanmeng Yan, Zijun Liu, Yanan Wu, Hong Xu, Huixing Jiang, Weiran Xu

Ouroboros: Generating Longer Drafts Phrase by Phrase for Faster Speculative Decoding (2024.emnlp-main)

Challenge:	Existing methods of OOD detection only focus on whether a sample is correctly classified . lack of real OOD examples leads to poor prior knowledge about these unknown intents .
Approach:	They propose a supervised contrastive learning objective to minimize intra-class variance . they employ an adversarial augmentation mechanism to obtain pseudo diverse views .
Outcome:	The proposed method minimizes intra-class variance by pulling together in-domain intents belonging to the same class and maximizes inter-class variation by pushing apart samples from different classes.

Copied to clipboard

Weilin Zhao, Yuxiang Huang, Xu Han, Wang Xu, Chaojun Xiao, Xinrong Zhang, Yewei Fang, Kaihuo Zhang, Zhiyuan Liu, Maosong Sun

Challenge:	Speculative decoding is a widely used method that accelerates the generation process of large language models (LLMs) drafting efficiency has become a bottleneck in the final speedup of speculative drafting, therefore generating longer drafts at less cost can lead to better speedup.
Approach:	They propose a method that uses existing model to drafting and target LLM to verify draft in a low-cost parallel manner.
Outcome:	The proposed method can achieve speedups of up to 2.4 over speculative decoding and 3.9 over vanilla decoding without fine-tuning draft and target models.

Emergent Modularity in Pre-trained Transformers (2023.findings-acl)

Copied to clipboard

Zhengyan Zhang, Zhiyuan Zeng, Yankai Lin, Chaojun Xiao, Xiaozhi Wang, Xu Han, Zhiyuan Liu, Ruobing Xie, Maosong Sun, Jie Zhou

CAR: Empowering Agents with Dynamic Tool Synthesis and Global Trajectory Rectification (2026.findings-acl)

Challenge:	Existing studies on pre-trained Transformers show that they learn fine-grained neuron functions.
Approach:	They examine the presence of modularity in pre-trained Transformers . they focus on Mixture-of-Experts, a promising candidate for modularity .
Outcome:	The proposed structure stabilizes at the early stage, which is faster than neuron stabilization.

Copied to clipboard

Kai Wang, Xu Wang, Zhiyuan Fu, Yudong Zhang, Yang Wang

Challenge:	Existing LLM agents are brittle in open-ended environments due to two limitations: 1) a closed action space; 2) myopic error recovery.
Approach:	They propose a novel architecture that augments the action space and revises global strategies by adding a reflective replanning mechanism to the system.
Outcome:	Experiments show that CAR outperforms baselines in a diagnostic benchmark with pruned toolsets to simulate tool scarcity.

FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluation (D18-1)

Copied to clipboard

Xu Han, Hao Zhu, Pengfei Yu, Ziyun Wang, Yuan Yao, Zhiyuan Liu, Maosong Sun

Low-Resource NLP

Challenge:	Empirical results show that even the most competitive few-shot learning models struggle on this task, especially as compared with humans.
Approach:	They propose a Few-Shot Relation Classification Dataset consisting of 70, 000 sentences on 100 relations derived from Wikipedia and annotated by crowdworkers.
Outcome:	The proposed methods perform well on the most competitive few-shot learning models, especially as compared with humans.

CLEVE: Contrastive Pre-training for Event Extraction (2021.acl-long)

Copied to clipboard

Ziqi Wang, Xiaozhi Wang, Xu Han, Yankai Lin, Lei Hou, Zhiyuan Liu, Peng Li, Juanzi Li, Jie Zhou

Be Careful about Poisoned Word Embeddings: Exploring the Vulnerability of the Embedding Layers in NLP Models (2021.naacl-main)

Challenge:	Existing EE methods do not model event characteristics from large unsupervised data.
Approach:	They propose a contrastive pre-training framework for event extraction to better learn event knowledge from large unsupervised data and their semantic structures.
Outcome:	The proposed framework improves on ACE 2005 and MAVEN datasets on event extraction tasks.

Copied to clipboard

Wenkai Yang, Lei Li, Zhiyuan Zhang, Xuancheng Ren, Xu Sun, Bin He

Representation Learning

Challenge:	Recent studies reveal a security threat to natural language processing models, called the Backdoor Attack.
Approach:	They propose to hack a model by modifying one single word embedding vector without sacrificing accuracy on clean samples.
Outcome:	The proposed method is more efficient and stealthier on sentiment analysis and sentence-pair classification tasks.

Adversarial Multi-lingual Neural Relation Extraction (C18-1)

Copied to clipboard

Xiaozhi Wang, Xu Han, Yankai Lin, Zhiyuan Liu, Maosong Sun

DeepNote: Note-Centric Deep Retrieval-Augmented Generation (2025.findings-emnlp)

Challenge:	Existing models cannot capture consistency and diversity of relation patterns in different languages.
Approach:	They propose an adversarial multi-lingual neural relation extraction model which considers consistency and diversity among languages.
Outcome:	The proposed model outperforms the state-of-the-art models on real-world datasets.

Copied to clipboard

Ruobing Wang, Qingfei Zhao, Yukun Yan, Daren Zha, Yuxuan Chen, Shi Yu, Zhenghao Liu, Yixuan Wang, Shuo Wang, Xu Han, Zhiyuan Liu, Maosong Sun

Information Retrieval

Challenge:	. - (EN)
Approach:	. - (EN)
Outcome:	. - (EN)

MatPlotAgent: Method and Evaluation for LLM-Based Agentic Scientific Data Visualization (2024.findings-acl)

Copied to clipboard

Zhiyu Yang, Zihan Zhou, Shuo Wang, Xin Cong, Xu Han, Yukun Yan, Zhenghao Liu, Zhixing Tan, Pengyuan Liu, Dong Yu, Zhiyuan Liu, Xiaodong Shi, Maosong Sun

Multimodality

Challenge:	Scientific data visualization is an essential process in research, but its use of large language models remains unexplored.
Approach:	They propose a model-agnostic LLM agent framework to automate scientific data visualization tasks.
Outcome:	The proposed framework improves performance of commercial and open-source models.

SimPBL: A Multi-Agent Framework for Project-Based Learning (2026.acl-long)

Copied to clipboard

Daniel Zhang-Li, Joy Jia Yin Lim, Binglin Liu, Shangqing Tu, Zijun Yao, Hao Peng, Jifan Yu, Haoxuan Li, Zhanxin Hao, Ye He, Zekun Li, Jiangyi Wang, Lei Hou, Bin Xu, Xin Cong, Zhiyuan Liu, Huiqin Liu, Yu Zhang, Juanzi Li

Challenge:	Existing LLMs provide partial assistance without modeling these roles, and overly comprehensive help can reduce learner autonomy.
Approach:	They propose a multi-agent framework with an orchestrator agent that provides adaptive scaffolding from interaction logs and collaborator agents that support project work through boundary-aware collaboration.
Outcome:	The proposed framework improves learner examination scores by 14% . it is based on a multi-agent framework with an orchestrator agent .

BMInf: An Efficient Toolkit for Big Model Inference and Tuning (2022.acl-demo)

Copied to clipboard

Xu Han, Guoyang Zeng, Weilin Zhao, Zhiyuan Liu, Zhengyan Zhang, Jie Zhou, Jun Zhang, Jia Chao, Maosong Sun

PPT: Pre-trained Prompt Tuning for Few-shot Learning (2022.acl-long)

Challenge:	Recent years, pre-trained language models (PLMs) have achieved promising results on various NLP tasks.
Approach:	They propose an open-source toolkit for big model inference and tuning which can support big model tuning at extremely low computation cost.
Outcome:	The proposed toolkit can support big model inference and tuning at extremely low computation cost.

Copied to clipboard

Yuxian Gu, Xu Han, Zhiyuan Liu, Minlie Huang

APB-V: Accelerating Long-Video Understanding via Sequence-Parallelism-aware Approximate Attention (2026.acl-long)

Challenge:	Prompt tuning for pre-trained language models has shown remarkable performance . however, prompt tuning is still not fully explored .
Approach:	They propose to pre-train prompts by adding soft prompts into the pre-training stage to obtain a better initialization.
Outcome:	The proposed framework outperforms full-model tuning under full-data and few-shot learning settings.

Copied to clipboard

Yuxiang Huang, Mingye Li, Xu Han, Chaojun Xiao, Weilin Zhao, Ao Sun, Ziqi Yuan, Hao Zhou, Fandong Meng, Zhiyuan Liu

FewRel 2.0: Towards More Challenging Few-Shot Relation Classification (D19-1)

Challenge:	Existing methods for long-video inference use compression or sparse attention . existing methods restrict LMMs from handling longer, more complex videos .
Approach:	They propose a sequence-parallel framework with optimized attention that accelerates long-video inference across multiple GPUs.
Outcome:	The proposed framework delivers speedups of 12.72x, 1.70x, and 1.18x over FlashAttn, ZigZagRing, and APB without significant performance loss.

Copied to clipboard

Tianyu Gao, Xu Han, Hao Zhu, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou

Enhancing Byzantine-Resistant Aggregations with Client Embedding (2024.findings-emnlp)

Challenge:	Few-shot domain adaptation and NOTA detection are two real-world challenges for few-shot relation classification models.
Approach:	They propose a task to investigate two aspects of few-shot relation classification models . they build upon the FewRel dataset by adding a new test set in a different domain .
Outcome:	The proposed task can evaluate few-shot domain adaptation and few- shot none-of-the-above detection on a new domain and NOTA relation choice.

Copied to clipboard

Zhiyuan Zhang, Hao Zhou, Fandong Meng, Jie Zhou, Xu Sun

Robustness in NLP

Challenge:	Existing Byzantine-resistant aggregations detect poisonous clients but cannot defend against backdoor injection by malicious attackers in natural language tasks.
Approach:	They propose to embed client parameters to enhance Byzantine-resistant aggregations.
Outcome:	The proposed client embeddings detect poisonous clients and discard them . the proposed algorithms can't defend against backdoor injection by malicious attackers in natural language tasks .

Cross-Lingual Contrastive Learning for Fine-Grained Entity Typing for Low-Resource Languages (2022.acl-long)

Copied to clipboard

Xu Han, Yuqi Luo, Weize Chen, Zhiyuan Liu, Maosong Sun, Zhou Botong, Hao Fei, Suncong Zheng

Open Hierarchical Relation Extraction (2021.naacl-main)

Challenge:	Experimental results show that by applying our framework, we can easily learn effective FGET models for low-resource languages.
Approach:	They propose a cross-lingual contrastive learning framework to learn FGET models for low-resource languages.
Outcome:	The proposed framework can learn effective FGET models for low-resource languages even without human-labeled data.

Copied to clipboard

Kai Zhang, Yuan Yao, Ruobing Xie, Xu Han, Zhiyuan Liu, Fen Lin, Leyu Lin, Maosong Sun

Dynamic Anticipation and Completion for Multi-Hop Reasoning over Sparse Knowledge Graph (2020.emnlp-main)

Challenge:	Existing OpenRE methods cast different relation types in isolation without considering their hierarchical dependency.
Approach:	They propose a framework to establish bidirectional connections between OpenRE and relation hierarchies by integrating hierarchy information into relation representations.
Outcome:	The proposed framework outperforms state-of-the-art models on relation clustering and hierarchy expansion.

Copied to clipboard

Xin Lv, Xu Han, Lei Hou, Juanzi Li, Zhiyuan Liu, Wei Zhang, Yichi Zhang, Hao Kong, Suhui Wu

Structured Data in NLP

Challenge:	Existing reasoning methods for sparse KGs are incomplete and lack of evidential paths to target entities makes multi-hop reasoning difficult.
Approach:	They propose a multi-hop reasoning model over sparse KGs to solve this problem . they use latent prediction of embedding-based models to make the model perform more potential path search over sparses .
Outcome:	The proposed method outperforms state-of-the-art models on five datasets from Freebase, NELL and Wikidata.

FR-Spec: Accelerating Large-Vocabulary Language Models via Frequency-Ranked Speculative Sampling (2025.acl-long)

Copied to clipboard

Weilin Zhao, Tengyu Pan, Xu Han, Yudi Zhang, Sun Ao, Yuxiang Huang, Kaihuo Zhang, Weilun Zhao, Yuxuan Li, Jie Zhou, Hao Zhou, Jianyong Wang, Maosong Sun, Zhiyuan Liu

MAVEN-ERE: A Unified Large-scale Dataset for Event Coreference, Temporal, Causal, and Subevent Relation Extraction (2022.emnlp-main)

Challenge:	Speculative sampling is an efficient way to accelerate the auto-regressive generation process of large language models.
Approach:	They propose a frequency-ranked speculative sampling framework that optimizes draft candidate selection through vocabulary space compression.
Outcome:	Experiments show that FR-Spec reduces LM Head computation overhead by 75% while ensuring the equivalence of the final output distribution.

Copied to clipboard

Xiaozhi Wang, Yulin Chen, Ning Ding, Hao Peng, Zimu Wang, Yankai Lin, Xu Han, Lei Hou, Juanzi Li, Zhiyuan Liu, Peng Li, Jie Zhou

Dim-Krum: Backdoor-Resistant Federated Learning for NLP with Dimension-wise Krum-Based Aggregation (2022.findings-emnlp)

Challenge:	Existing datasets only cover limited relation types at once, which prevents models from taking full advantage of relation interactions.
Approach:	They construct a large-scale human-annotated ERE dataset with improved annotation schemes to address these drawbacks.
Outcome:	The proposed dataset is larger than existing datasets of all the ERE tasks by at least an order of magnitude.

Copied to clipboard

Zhiyuan Zhang, Qi Su, Xu Sun

Robustness in NLP

Challenge:	Existing methods to reduce backdoor risk of federated learning are validated in the CV field.
Approach:	They propose a federated aggregation algorithm that detects errors determined by backdoor strengths for NLP attacks.
Outcome:	The proposed method is hard to defend against than CV, and the results validate it.

ThinkNote: Enhancing Knowledge Integration and Utilization of Large Language Models via Constructivist Cognition Modeling (2026.findings-eacl)

Copied to clipboard

Zhipeng Xu, Zhenghao Liu, Yukun Yan, Shuo Wang, Shi Yu, Zheni Zeng, Chaojun Xiao, Zhiyuan Liu, Ge Yu, Chenyan Xiong

AutoReproduce: Automatic AI Experiment Reproduction with Paper Lineage (2026.acl-long)

Challenge:	Large Language Models (LLMs) exhibit suboptimal behaviors and inconsistencies when exposed to unfamiliar external information, underscoring their limitations in effectively leveraging such knowledge.
Approach:	They propose a framework that enhances the external knowledge utilization of Large Language Models through a two-stage constructivist cognitive modeling process.
Outcome:	The proposed framework achieves a 10% improvement over baseline methods on various question-answering benchmarks.

Copied to clipboard

Xuanle Zhao, Zilin Sang, Yuxuan Li, Qi Shi, Weilun Zhao, Shuo Wang, Duzhen Zhang, Xu Han, Zhiyuan Liu, Maosong Sun

Challenge:	Efficient reproduction of research papers requires deep domain expertise.
Approach:	They propose a framework that systematically mines implicit knowledge from the cited literature to reproduce experimental code in a complete, end-to-end manner.
Outcome:	The proposed framework surpasses baselines across all metrics and reproduces experimental code in a complete, end-to-end manner.

Robust and Scalable Model Editing for Large Language Models (2024.lrec-main)

Copied to clipboard

Yingfa Chen, Zhengyan Zhang, Xu Han, Chaojun Xiao, Zhiyuan Liu, Chen Chen, Kuai Li, Tao Yang, Maosong Sun

Open Relation Extraction: Relational Knowledge Transfer from Supervised Data to Unsupervised Data (D19-1)

Challenge:	Existing methods that ignore contextual knowledge fail to reliably fall back to parametric knowledge when presented with irrelevant context.
Approach:	They propose to use contextual knowledge to update and correct LLMs' knowledge by in-context editing instead of retraining.
Outcome:	The proposed method outperforms current state-of-the-art methods by a large margin on a dataset that contains irrelevant questions.

Copied to clipboard

Ruidong Wu, Yuan Yao, Xu Han, Ruobing Xie, Zhiyuan Liu, Fen Lin, Leyu Lin, Maosong Sun

LoRE: Enhancing Search Relevance with Progressive Chain-of-Thought and Preference Alignment (2026.findings-acl)

Challenge:	Existing methods to extract relational facts from open domain corpora are time-consuming and human-intensive.
Approach:	They propose a framework to learn similarity metrics of relations from labeled data . they propose to transfer relational knowledge to identify novel relations in unlabeled data.
Outcome:	Experiments on two real-world datasets show that the proposed framework improves compared with state-of-the-art methods.

Copied to clipboard

Chenji Lu, Zhuo Chen, Hui Zhao, Zhiyuan Zeng, Gang Zhao, Junjie Ren, null Lihaoran, Songyan Liu, Pengjie Wang, Chuan Yu, Jian Xu, Bo Zheng

Reasoning

Challenge:	E-commerce search relevance is a critical component of retrieval systems.
Approach:	They propose a large-generative model for search relevance that trains reasoning knowledge, multi-modal understanding and rule awareness into three core competencies.
Outcome:	The proposed model outperforms GPT-5 in Macro-F1 and achieves 27% online gain.

DIAG-NRE: A Neural Pattern Diagnosis Framework for Distantly Supervised Neural Relation Extraction (P19-1)

Copied to clipboard

Shun Zheng, Xu Han, Yankai Lin, Peilin Yu, Lu Chen, Ling Huang, Zhiyuan Liu, Wei Xu

Plug-and-Play Knowledge Injection for Pre-trained Language Models (2023.acl-long)

Challenge:	Existing methods for labeling relational facts require significant expert labor to write relation-specific patterns, which makes them too sophisticated to generalize quickly.
Approach:	They propose a neural pattern diagnosis framework that can summarize and refine relation-specific patterns with human experts in the loop.
Outcome:	The proposed framework can summarize and refine high-quality relational patterns from noise data with human experts in the loop.

Copied to clipboard

Zhengyan Zhang, Zhiyuan Zeng, Yankai Lin, Huadong Wang, Deming Ye, Chaojun Xiao, Xu Han, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou

OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problems (2024.acl-long)

Challenge:	Existing knowledge injection methods are not suitable for enhancing pre-trained language models with external knowledge bases.
Approach:	They propose a plug-and-play knowledge injection method where knowledge bases are injected into frozen existing downstream models by a knowledge plugin.
Outcome:	The proposed method improves the performance of knowledge injection on knowledge-driven tasks while keeping model parameters frozen.

Copied to clipboard

Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, Jie Liu, Lei Qi, Zhiyuan Liu, Maosong Sun

GA-SAM: Gradient-Strength based Adaptive Sharpness-Aware Minimization for Improved Generalization (2022.emnlp-main)

Challenge:	Large Language Models (LLMs) and Large Multimodal Models have exceeded general human capabilities in various tasks.
Approach:	They present an Olympiad-level bilingual multimodal scientific benchmark featuring 8,476 problems from Olympiad level mathematics and physics competitions.
Outcome:	The best performing model, GPT-4V, attains an average score of 17.97% on OlympiadBench, with a mere 10.74% in physics, highlighting the benchmark rigor and the intricacy of physical reasoning.

Copied to clipboard

Zhiyuan Zhang, Ruixuan Luo, Qi Su, Xu Sun

BMCook: A Task-agnostic Compression Toolkit for Big Models (2022.emnlp-demos)

Challenge:	Recent studies show flat minima tend to imply better generalization abilities . however, it has some difficulty implying SAM to some natural language tasks .
Approach:	They propose a flatness-aware minimization algorithm that can be applied to natural language tasks . they propose to use parameter corruptions to explain why flat minima generalize better .
Outcome:	The proposed algorithm can generalize better for flat minima that are robust against corruptions or perturbations.

Copied to clipboard

Zhengyan Zhang, Baitao Gong, Yingfa Chen, Xu Han, Guoyang Zeng, Weilin Zhao, Yanxu Chen, Zhiyuan Liu, Maosong Sun

Boosting Inference Efficiency: Unleashing the Power of Parameter-Shared Pre-trained Language Models (2023.findings-emnlp)

Challenge:	Existing efforts to compress medium-sized models for specific tasks have limited results.
Approach:	They propose a task-agnostic compression toolkit for big models that implements quantization, pruning, distillation and MoEfication methods.
Outcome:	The proposed tool improves performance on a model with 3 billion parameters by 12x . it also outperforms the original model on three typical NLP benchmarks.

Copied to clipboard

Weize Chen, Xiaoyue Xu, Xu Han, Yankai Lin, Ruobing Xie, Zhiyuan Liu, Maosong Sun, Jie Zhou

Cost-Optimal Grouped-Query Attention for Long-Context Modeling (2025.emnlp-main)

Challenge:	Parameter-shared pre-trained language models (PLMs) have emerged as a successful approach in resource-constrained environments.
Approach:	They propose a method to enhance the inference efficiency of parameter-shared PLMs by pre-training models that can achieve even greater acceleration.
Outcome:	The proposed method improves inference efficiency on autoregressive and autoencoding models.

Copied to clipboard

Yingfa Chen, Yutong Wu, Chenyang Song, Zhen Leng Thai, Xingyu Shen, Xu Han, Zhiyuan Liu, Maosong Sun

Integrating Representation Subspace Mapping with Unimodal Auxiliary Loss for Attention-based Multimodal Emotion Recognition (2024.lrec-main)

Challenge:	Current GQA configurations overlook how context length influences inference cost .
Approach:	They propose a recipe for deriving cost-optimal GQA configurations that decouple the total head size from the hidden size and allow more flexible control over attention FLOPs.
Outcome:	The proposed configurations reduce memory usage and FLOPs by more than 50% compared to Llama-3's GQA, with no degradation in model capabilities.

Copied to clipboard

Xulong Du, Xingnan Zhang, Dandan Wang, Yingying Xu, Zhiyuan Wu, Shiqing Zhang, Xiaoming Zhao, Jun Yu, Liangliang Lou

Sentiment Analysis

Challenge:	Existing methods to identify emotions rely on a large modality gap in their representations .
Approach:	They propose a representation subspace mapping module that maps each modality into two distinct subspaces and a cross-modality attention module that leverages auxiliary loss to remove the noise unrelated to emotion classification.
Outcome:	The proposed approach achieves superior performance to state-of-the-art MER methods on the IEMOCAP and MSP-Improv datasets.

Pretrain-KGE: Learning Knowledge Representation from Pretrained Language Models (2020.findings-emnlp)

Copied to clipboard

Zhiyuan Zhang, Xiaoqian Liu, Yi Zhang, Qi Su, Xu Sun, Bin He

Neural Network Surgery: Injecting Data Patterns into Pre-trained Models with Minimal Instance-wise Side Effects (2021.naacl-main)

Challenge:	Existing knowledge graph embedding models suffer from limited knowledge representation due to sparse and noisy dataset annotations.
Approach:	They propose to use pretrained language models to enhance knowledge representation by leveraging world knowledge from pretrained models.
Outcome:	Extensive experiments show that the proposed framework can improve results over existing models.

Copied to clipboard

Zhiyuan Zhang, Xuancheng Ren, Qi Su, Xu Sun, Bin He

WebCPM: Interactive Web Search for Chinese Long-form Question Answering (2023.acl-long)

Challenge:	Existing neural network tuning methods cause instance-wise side effects . et al., 2018: a new approach to perform neural network surgery .
Approach:	They propose to perform neural network surgery by only changing 10-5 parameters . they propose to use a dynamic selecting method to achieve the best overall performance .
Outcome:	The proposed method achieves the best overall performance and induces fewer instance-wise side effects by changing only 10-5 of the parameters.

Copied to clipboard

Yujia Qin, Zihan Cai, Dian Jin, Lan Yan, Shihao Liang, Kunlun Zhu, Yankai Lin, Xu Han, Ning Ding, Huadong Wang, Ruobing Xie, Fanchao Qi, Zhiyuan Liu, Maosong Sun, Jie Zhou

Information Retrieval

Challenge:	Long-form question answering requires two procedures: information retrieval and information synthesis.
Approach:	They propose a Chinese long-form question answering dataset called WebCPM . the dataset is based on a web search interface that engages with a search engine in real time .
Outcome:	The proposed dataset generates answers that are no worse than human-written ones . the dataset is the first Chinese LFQA dataset .

Expose Backdoors on the Way: A Feature-Based Efficient Defense against Textual Backdoor Attacks (2022.findings-emnlp)

Copied to clipboard

Sishuo Chen, Wenkai Yang, Zhiyuan Zhang, Xiaohan Bi, Xu Sun

MAgIC: Investigation of Large Language Model Powered Multi-Agent in Cognition, Adaptability, Rationality and Collaboration (2024.emnlp-main)

Challenge:	Existing online backdoor defense methods for NLP models focus on anomalies at input or output level, causing fragility to adaptive attacks and high computational cost.
Approach:	They propose a feature-based online defense method to detect poisoned samples . they use a distance-based anomaly score to distinguish poisones from clean samples based on feature-level regularization .
Outcome:	The proposed method outperforms existing methods in sentiment analysis and offense detection tasks.

Copied to clipboard

Lin Xu, Zhiyuan Hu, Daquan Zhou, Hongyu Ren, Zhen Dong, Kurt Keutzer, See-Kiong Ng, Jiashi Feng

Manual Evaluation Matters: Reviewing Test Protocols of Distantly Supervised Relation Extraction (2021.findings-acl)

Challenge:	Large language models (LLMs) have advanced natural language processing, demonstrating exceptional reasoning, tool usage, and memory capabilities.
Approach:	They propose a competition-based benchmark framework specifically designed to assess LLMs within multi-agent environments.
Outcome:	The proposed framework enhances the LLMs’ abilities in navigating complex social and cognitive dimensions by over threefold between the strongest and weakest LLM models.

Copied to clipboard

Tianyu Gao, Xu Han, Yuzhuo Bai, Keyue Qiu, Zhiyu Xie, Yankai Lin, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou

ChatVLA: Unified Multimodal Understanding and Robot Control with Vision-Language-Action Model (2025.emnlp-main)

Challenge:	Distantly supervised relation extraction (RE) has attracted much attention in the past few years . previous methods to evaluate models manually or directly on autolabeled data have produced inaccurate evaluations .
Approach:	They propose to use distant supervision to generate large-scale autolabeled data . they build manually-annotated test sets for two DS-RE datasets and evaluate models .
Outcome:	The proposed method produces 53% wrong labels at the entity pair level in the popular NYT10 dataset.

Copied to clipboard

Zhongyi Zhou, Yichen Zhu, Minjie Zhu, Junjie Wen, Ning Liu, Zhiyuan Xu, Weibin Meng, Yaxin Peng, Chaomin Shen, Feifei Feng, Yi Xu

Multimodality

Challenge:	Recent advances in vision-language-action models prioritize robotic action mastery . however, models trained on visual-text pairs struggle to interpret multimodal data .
Approach:	They propose a framework that integrates multimodal data after initial control mastery and a Mixture-of-Experts architecture to minimize task interference.
Outcome:	The proposed framework surpasses state-of-the-art vision-language-action (VLA) methods on multimodal understanding benchmarks and achieves six times higher performance on visual question-answering datasets.

Beyond the Turn-Based Game: Enabling Real-Time Conversations with Duplex Models (2024.emnlp-main)

Copied to clipboard

Xinrong Zhang, Yingfa Chen, Shengding Hu, Xu Han, Zihang Xu, Yuanwei Xu, Weilin Zhao, Maosong Sun, Zhiyuan Liu

Dialogue Systems & Conversational Agents

Challenge:	Large language models (LLMs) are increasingly permeating daily lives and require real-time interactions that mirror human conversations.
Approach:	They propose to use time-division-multiplexing to process queries and responses pseudo-simultaneously.
Outcome:	The proposed model can listen to users while generating output and adjust to provide instant feedback.

Beyond ’Aha!’: Toward Systematic Meta-Abilities Alignment in Large Reasoning Models (2026.findings-acl)

Copied to clipboard

Zhiyuan Hu, Yibo Wang, Hanze Dong, Yuhui Xu, Amrita Saha, Caiming Xiong, Bryan Hooi, Junnan Li

Reasoning

Challenge:	Prior work has shown that outcome-based reinforcement learning (RL) can incidentally elicit advanced reasoning behaviors such as self-correction, backtracking, and verification.
Approach:	They explicitly align large reasoning models with three meta-abilities: deduction, induction, and abduction, using automatically generated, self-verifiable tasks.
Outcome:	The proposed model aligns models with deduction, induction, and abduction meta-abilities using automatically generated, self-verifiable tasks.

UltraEval: A Lightweight Platform for Flexible and Comprehensive Evaluation for LLMs (2024.acl-demos)

Copied to clipboard

Chaoqun He, Renjie Luo, Shengding Hu, Ranchi Zhao, Jie Zhou, Hanghao Wu, Jiajie Zhang, Xu Han, Zhiyuan Liu, Maosong Sun

Weize Chen, Hao Zhu, Xu Han, Zhiyuan Liu, Maosong Sun

Challenge:	Existing evaluation platforms are complex and poorly modularized, hindering seamless incorporation into researcher’s workflows.
Approach:	They propose a lightweight evaluation framework characterized by lightweight, comprehensiveness, modularity, and efficiency that integrates models, data, and metrics into a unified evaluation workflow.
Outcome:	The proposed evaluation framework is lightweight, comprehensive, modular, and efficient.

Copied to clipboard

SConU: Selective Conformal Uncertainty in Large Language Models (2025.acl-long)

Challenge:	a conceptually simple and effective method to quantify the similarity between relations is presented . identifying relations is a crucial problem for several information extraction tasks.
Approach:	They propose a method to quantify the similarity between relations in knowledge bases . they use a neural network to parameterize conditional probability distributions over entity pairs .
Outcome:	The proposed method significantly correlates with human judgments, the authors show . it could be incorporated into negative sampling and softmax classification to alleviate these mistakes.

Copied to clipboard

Zhiyuan Wang, Qingni Wang, Yue Zhang, Tianlong Chen, Xiaofeng Zhu, Xiaoshuang Shi, Kaidi Xu

Fully Hyperbolic Neural Networks (2022.acl-long)

Challenge:	Existing frameworks fail to identify outliers that violate the exchangeability assumption, leading to unbounded miscoverage rates and unactionable prediction sets.
Approach:	They propose a method that implements significance tests to determine whether a given sample deviates from the uncertainty distribution of the calibration set.
Outcome:	The proposed approach facilitates rigorous management of miscoverage rates across single-domain and interdisciplinary contexts, and enhances the efficiency of predictions.

Copied to clipboard

Weize Chen, Xu Han, Yankai Lin, Hexu Zhao, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou

Challenge:	Existing hyperbolic neural networks encode features in the hyperbolical space yet formalize most of their operations in the tangent space.
Approach:	They propose a fully hyperbolic framework to build hyperbolical networks based on the Lorentz model by adapting Lorentzer transformations to formalize essential operations of neural networks.
Outcome:	The proposed framework has better performance on four NLP tasks compared with existing hyperbolic models .

Variator: Accelerating Pre-trained Models with Plug-and-Play Compression Modules (2023.findings-emnlp)

Copied to clipboard

Chaojun Xiao, Yuqi Luo, Wenbin Zhang, Pengle Zhang, Xu Han, Yankai Lin, Zhengyan Zhang, Ruobing Xie, Zhiyuan Liu, Maosong Sun, Jie Zhou

Revisit Overconfidence for OOD Detection: Reassigned Contrastive Learning with Adaptive Class-dependent Threshold (2022.naacl-main)

Challenge:	Large language models (LLMs) have been successful on NLP tasks but require huge parameter sizes and computational resources.
Approach:	They propose a parameter-efficient acceleration method that enhances computational efficiency through plug-and-play compression plugins.
Outcome:	The proposed method saves 53% computational costs using only 0.9% additional parameters with a performance drop of less than 2%.

Copied to clipboard

Yanan Wu, Keqing He, Yuanmeng Yan, QiXiang Gao, Zhiyuan Zeng, Fujia Zheng, Lulu Zhao, Huixing Jiang, Wei Wu, Weiran Xu

Prompt-learning for Fine-grained Entity Typing (2022.findings-emnlp)

Challenge:	Existing methods for OOD detection are based on labeled in-domain data . detecting out-of-domain (OOD) or unknown intents is challenging .
Approach:	They propose a novel reassigned contrastive learning method to discriminate IND intents for over-confident OOD and an adaptive class-dependent local threshold mechanism to separate similar IND and OOD intents.
Outcome:	The proposed method is effective for both aspects of overconfidence issues.

Copied to clipboard

Ning Ding, Yulin Chen, Xu Han, Guangwei Xu, Xiaobin Wang, Pengjun Xie, Haitao Zheng, Zhiyuan Liu, Juanzi Li, Hong-Gee Kim

Denoising Relation Extraction from Document-level Distant Supervision (2020.emnlp-main)

Challenge:	Extensive experiments on fine-grained entity typing under fully supervised, few-shot, and zero-shot settings show the effectiveness of prompt-learning.
Approach:	They propose a prompt-learning pipeline that stimulates versatile knowledge of pre-trained language models (PLMs) by constructing entity-oriented verbalizers and templates and conducting masked language modeling.
Outcome:	The proposed approach can be applied to fine-grained entity typing in fully supervised, few-shot, and zero-shot scenarios.

Copied to clipboard

Chaojun Xiao, Yuan Yao, Ruobing Xie, Xu Han, Zhiyuan Liu, Maosong Sun, Fen Lin, Leyu Lin

HMEAE: Hierarchical Modular Event Argument Extraction (D19-1)

Challenge:	Existing methods to generate auto-labeled sentences for relation extraction (RE) are difficult to extend to document-level relation extraction as noise from DS may be even multiplied in documents.
Approach:	They propose a pre-trained model which de-emphasizes noisy DS data via multiple pre-training tasks.
Outcome:	The proposed model can capture useful information from noisy data and achieve promising results on the large-scale DocRE benchmark.

Copied to clipboard

Xiaozhi Wang, Ziqi Wang, Xu Han, Zhiyuan Liu, Juanzi Li, Peng Li, Maosong Sun, Jie Zhou, Xiang Ren

Progressive Self-Training with Discriminator for Aspect Term Extraction (2021.emnlp-main)

Challenge:	Existing event extraction methods classify each argument role independently, ignoring conceptual correlations between different argument roles.
Approach:	They propose a Hierarchical Modular Event Argument Extraction model to provide inductive bias from the concept hierarchy of event argument roles.
Outcome:	The proposed model outperforms existing methods on real-world datasets and shows that it leverages useful knowledge from the concept hierarchy.

Copied to clipboard

Qianlong Wang, Zhiyuan Wen, Qin Zhao, Min Yang, Ruifeng Xu

Fine-Grained Legal Argument-Pair Extraction via Coarse-Grained Pre-training (2024.lrec-main)

Challenge:	Existing approaches to extract aspect terms from review sentences are limited due to lack of annotated data.
Approach:	They propose to refine conventional self-training to progressive self-teaching to reduce noise . they use a discriminator to filter the noisy pseudo-labels.
Outcome:	The proposed model outperforms baseline models and achieves state-of-the-art performance on four SemEval datasets.

Copied to clipboard

Chaojun Xiao, Yutao Sun, Yuan Yao, Xu Han, Wenbin Zhang, Zhiyuan Liu, Maosong Sun

Adversarial Self-Supervised Learning for Out-of-Domain Detection (2021.naacl-main)

Challenge:	Current methods conceptualize LAE as a supervised sentence-pair classification problem and necessitate extensive manual annotations.
Approach:	They propose a model that focuses on fine-grained alignment of argument pairs building upon coarse-grain complaint-defense pairs.
Outcome:	The proposed model outperforms baseline models by 3.7 and 2.4 points on average.

Copied to clipboard

Zhiyuan Zeng, Keqing He, Yuanmeng Yan, Hong Xu, Weiran Xu

UltraLink: An Open-Source Knowledge-Enhanced Multilingual Supervised Fine-tuning Dataset (2024.acl-long)

Challenge:	Existing methods for detecting out-of-domain (OOD) intents are unsupervised and require extensive labeled data.
Approach:	They propose a self-supervised contrastive learning framework to model discriminative semantic features from unlabeled data.
Outcome:	The proposed framework outperforms baseline methods on two public benchmark datasets with a statistically significant margin.

Copied to clipboard

Haoyu Wang, Shuo Wang, Yukun Yan, Xujia Wang, Zhiyu Yang, Yuzhuang Xu, Zhenghao Liu, Liner Yang, Ning Ding, Xu Han, Zhiyuan Liu, Maosong Sun

Hierarchical Relation Extraction with Coarse-to-Fine Grained Attention (D18-1)

Challenge:	Open-source large language models (LLMs) have gained strength across diverse fields, but the majority of studies focus on English.
Approach:	They propose a knowledge-grounded data augmentation approach to elicit more language-specific knowledge of LLMs by enhancing their ability to serve users from different countries.
Outcome:	The proposed method can prune the language-agnostic supervised fine-tuning dataset without any performance degradation.

Copied to clipboard

Xu Han, Pengfei Yu, Zhiyuan Liu, Maosong Sun, Peng Li

Enhancing Chat Language Models by Scaling High-quality Instructional Conversations (2023.emnlp-main)

Challenge:	Existing methods for relation extraction use knowledge graphs to automatically label training data . but, it suffers from the wrong labeling problem because not all sentences containing two entities can express their relations in KGs .
Approach:	They propose a distant supervision approach to automatically label training instances . they integrate hierarchical information of relations into distantly supervised relation extraction .
Outcome:	The proposed model outperforms baseline models on a large-scale dataset.

Copied to clipboard

Ning Ding, Yulin Chen, Bokai Xu, Yujia Qin, Shengding Hu, Zhiyuan Liu, Maosong Sun, Bowen Zhou

Migician: Revealing the Magic of Free-Form Multi-Image Grounding in Multimodal Large Language Models (2025.findings-acl)

Challenge:	a recent study validates the effectiveness of chat language models by fine-tuning instruction data.
Approach:	They propose to use a large-scale dataset of instructional conversations to fine-tune a conversational model on instruction data.
Outcome:	The proposed model outperforms open-source models in key metrics including scale, average length, diversity, coherence, etc.

Copied to clipboard

You Li, Heyu Huang, Chi Chen, Kaiyu Huang, Chao Huang, Zonghao Guo, Zhiyuan Liu, Jinan Xu, Yuhua Li, Ruixuan Li, Maosong Sun

Multimodality

Challenge:	Existing MLLMs still struggle to achieve precise grounding in multi-image scenarios.
Approach:	They propose a Chain-of-Thought framework that integrates single-image grounding with multi-image comprehension to address this challenge.
Outcome:	The proposed model outperforms existing models in multi-image grounding tasks by 24.94% and surpasses larger 70B models.

LM-Interview: An Easy-to-use Smart Interviewer System via Knowledge-guided Language Model Exploitation (2024.emnlp-demo)

Copied to clipboard

Hanming Li, Jifan Yu, Ruimiao Li, Zhanxin Hao, Yan Xuan, Jiaxi Yuan, Bin Xu, Juanzi Li, Zhiyuan Liu

RealMem: Benchmarking LLMs in Real-World Memory-Driven Interaction (2026.findings-acl)

Challenge:	Semi-structured interviews are a crucial method of data acquisition in qualitative research.
Approach:	They propose a semi-structured interview system that automates interview preparation, analysis and control by interviewers.
Outcome:	Experimental results show that LM-Interview performs comparable to human interviewers . the system can be used to analyze semi-structured interviews without interviewers' involvement .

Copied to clipboard

Haonan Bian, Zhiyuan Yao, Sen Hu, Zishan Xu, Shaolei Zhang, Yifu Guo, Ziliang Yang, Xueran Han, Huacan Wang, Ronghao Chen

Dialogue Systems & Conversational Agents

Challenge:	Existing benchmarks focus on casual conversation or task-oriented dialogue, failing to capture “long-term project-oriented” interactions where agents must track evolving goals.
Approach:	They propose a benchmark that simulates the dynamic evolution of memory in real-world projects.
Outcome:	The proposed benchmarks simulate the dynamic evolution of memory in real-world projects.

Continual Relation Learning via Episodic Memory Activation and Reconsolidation (2020.acl-main)

Copied to clipboard

Xu Han, Yi Dai, Tianyu Gao, Yankai Lin, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou

Challenge:	Existing methods to learn incessantly emerging novel relations are overfitting the few memorized examples of old relations, causing confusion among existing relations.
Approach:	They introduce episodic memory activation and reconsolidation (EMAR) to continual relation learning.
Outcome:	The proposed method outperforms state-of-the-art models in catastrophic forgetting old relations.

ConU: Conformal Uncertainty in Large Language Models with Correctness Coverage Guarantees (2024.findings-emnlp)

Copied to clipboard

Zhiyuan Wang, Jinhao Duan, Lu Cheng, Yue Zhang, Qingni Wang, Xiaoshuang Shi, Kaidi Xu, Heng Tao Shen, Xiaofeng Zhu

Multi-Modal Multi-Granularity Tokenizer for Chu Bamboo Slips (2025.coling-main)

Challenge:	Uncertainty quantification (UQ) in natural language generation tasks remains an open challenge . however, black-box uncertainty measures require investigating with the proliferation of LLMs served via APIs.
Approach:	They propose a conformal uncertainty measure and a method to transform heuristic uncertainty notions into rigorous prediction sets.
Outcome:	Empirical results show that the proposed method outperforms state-of-the-art methods and can provide reliable guarantees for open-ended NLG tasks.

Copied to clipboard

Yingfa Chen, Chenlong Hu, Cong Feng, Chenyang Song, Shi Yu, Xu Han, Zhiyuan Liu, Maosong Sun

Syntactic Text Processing

Challenge:	Using a multi-modal multi-granularity tokenizer, we analyze ancient Chinese scripts . a large proportion of the characters in ancient Chinese are rare or undeciphered .
Approach:	They propose a multi-modal multi-granularity tokenizer specifically designed for ancient Chinese scripts.
Outcome:	The proposed tokenizer improves on the part-of-speech tagging task on the Chu bamboo slip script.

Gradient-Based Adversarial Factual Consistency Evaluation for Abstractive Summarization (2021.emnlp-main)

Copied to clipboard

Zhiyuan Zeng, Jiaze Chen, Weiran Xu, Lei Li

OpenKE: An Open Toolkit for Knowledge Embedding (D18-2)

Challenge:	Abstractive summarization models often produce inconsistent statements or false facts.
Approach:	They propose an efficient weak-supervised adversarial data augmentation approach to generate factual consistency datasets by backpropagating gradients on token embeddings.
Outcome:	The proposed model can make interpretable factual errors tracing on public datasets and is cost-effective.

Copied to clipboard

Xu Han, Shulin Cao, Xin Lv, Yankai Lin, Zhiyuan Liu, Maosong Sun, Juanzi Li

Put It Back: Entity Typing with Language Model Enhancement (D18-1)

Challenge:	Existing knowledge embedding tools are available for embeddable knowledge graphs.
Approach:	They propose a unified framework and various fundamental models to embed knowledge graphs into a continuous low-dimensional space.
Outcome:	The toolkit and pre-trained embeddings are available on http://openke.thunlp.org/.

Copied to clipboard

Ji Xin, Hao Zhu, Xu Han, Zhiyuan Liu, Maosong Sun

StateX: Enhancing RNN Recall via Post-training State Expansion (2026.findings-acl)

Challenge:	Existing Entity typing models suffer from noisy labels due to distant supervision .
Approach:	They propose to enhance existing entity typing models with language model enhancement to measure compatibility between context sentences and labels.
Outcome:	The proposed model significantly outperforms the state-of-the-art model on benchmark datasets and is available on github.

Copied to clipboard

Xingyu Shen, Yingfa Chen, Zhen Leng Thai, Xu Han, Zhiyuan Liu, Maosong Sun

Building an Ellipsis-aware Chinese Dependency Treebank for Web Text (L18-1)

Challenge:	Existing studies show that RNNs with large recurrent states are expensive to train . however, the ability to recall contextual information from long contexts is underperforms them in certain aspects.
Approach:	They propose a framework that expands the states of pre-trained RNNs by scaling them up to 1.3B . they use a recurrent architecture that compresses contextual information into a fixedsize state .
Outcome:	Experiments on models with up to 1.3B parameters show that StateX expands state sizes without incurring high post-training costs or compromising other capabilities.

Copied to clipboard

Xuancheng Ren, Xu Sun, Ji Wen, Bingzhen Wei, Weidong Zhan, Zhiyuan Zhang

Syntactic Text Processing

Challenge:	ellipsis is a common linguistic phenomenon that some words are left out as they are understood from the context, especially in oral utterance.
Approach:	They propose to use a Chinese dependency treebank to facilitate the parsing of web text . they propose to restore omissions and reserve contexts in the web text to improve dependency parsers .
Outcome:	The proposed framework enables the parsing of web text from online microblogs.

Different Tunes Played with Equal Skill: Exploring a Unified Optimization Subspace for Parameter-Efficient Tuning (2022.findings-emnlp)

Copied to clipboard

Jing Yi, Weize Chen, Yujia Qin, Yankai Lin, Ning Ding, Xu Han, Zhiyuan Liu, Maosong Sun, Jie Zhou

Distribution Calibration for Out-of-Domain Detection with Bayesian Approximation (2022.coling-1)

Challenge:	Existing delta tuning algorithms freeze most of the parameters and only optimize minimal adaptive parameters.
Approach:	They propose to decompose DETs into a unified optimization subspace and conduct optimization within the subspace.
Outcome:	The proposed DETs achieve comparable performance to the original DET and can be transferred to another DET with non-trivial performance.

Copied to clipboard

Yanan Wu, Zhiyuan Zeng, Keqing He, Yutao Mou, Pei Wang, Weiran Xu

Challenge:	Existing methods for OOD detection are overconfident for OD samples . lack of labeled OOD examples leads to poor prior knowledge about these unknown intents, making it challenging to detect OOD samples.
Approach:	They propose a Bayesian OOD detection framework to calibrate distribution uncertainty using Monte-Carlo Dropout.
Outcome:	The proposed framework gains 33.33% OOD F1 improvements with increasing only 0.41% inference time compared to previous methods.

LLM×MapReduce: Simplified Long-Sequence Processing using Large Language Models (2025.acl-long)

Copied to clipboard

Zihan Zhou, Chong Li, Xinyi Chen, Shuo Wang, Yu Chao, Zhili Li, Haoyu Wang, Qi Shi, Zhixing Tan, Xu Han, Xiaodong Shi, Zhiyuan Liu, Maosong Sun

Learning from Context or Names? An Empirical Study on Neural Relation Extraction (2020.emnlp-main)

Challenge:	Existing studies have focused on extending the context length of large language models (LLMs) due to their quadratic computational complexity and a lack of high-quality long training examples, most LLMs are trained with a limited window size.
Approach:	They propose a training-free framework that enables large language models to effectively process long texts using a divide-and-conquer strategy for comprehensive document understanding.
Outcome:	The proposed framework outperforms open-source and commercial long-context LLMs and is compatible with several models.

Copied to clipboard

Hao Peng, Tianyu Gao, Xu Han, Yankai Lin, Peng Li, Zhiyuan Liu, Maosong Sun, Jie Zhou

Plug-and-Play Document Modules for Pre-trained Models (2023.acl-long)

Challenge:	Existing datasets may leak shallow heuristics via entity mentions, thus contributing to the high performance on RE benchmarks.
Approach:	They propose an entity-masked contrastive framework for relation extraction to gain a deeper understanding on textual context and type information while avoiding rote memorization of entities.
Outcome:	The proposed framework improves the effectiveness and robustness of neural models in different RE scenarios.

Copied to clipboard

Chaojun Xiao, Zhengyan Zhang, Xu Han, Chi-Min Chan, Yankai Lin, Zhiyuan Liu, Xiangyang Li, Zhonghua Li, Zhao Cao, Maosong Sun

ERNIE: Enhanced Language Representation with Informative Entities (P19-1)

Challenge:	Large-scale pre-trained models have been widely adopted for document-oriented NLP tasks, such as question answering.
Approach:	They propose to decouple document encoding from downstream tasks by introducing a document plugin into the backbone of a PTM.
Outcome:	The proposed model can encode documents once and for all across different scenarios.

Copied to clipboard

Zhengyan Zhang, Xu Han, Zhiyuan Liu, Xin Jiang, Maosong Sun, Qun Liu

IsOBS: An Information System for Oracle Bone Script (2020.emnlp-demos)

Challenge:	Existing pre-trained language models rarely consider incorporating knowledge graphs (KGs) Existing models capture rich semantic patterns from plain text and can be fine-tuned to improve performance of NLP tasks.
Approach:	They propose to incorporate knowledge graphs into pre-trained language models to enhance language representation with external knowledge.
Outcome:	The proposed model can take full advantage of lexical, syntactic, and knowledge information simultaneously.

Copied to clipboard

Xu Han, Yuzhuo Bai, Keyue Qiu, Zhiyuan Liu, Maosong Sun

Challenge:	Oracle bone script (OBS) documents are the oldest continuously-used writing system in the world and are important for linguistic and historical research.
Approach:	They construct an information system for OBS to symbolize, serialize, and store OBS data at the character-level using efficient databases and retrieval modules.
Outcome:	The proposed system symbolizes, serializes, and stores OBS data at the character-level, based on efficient databases and retrieval modules.

ACE-Router: Generalizing History-Aware Routing from MCP Tools to the Agent Web (2026.acl-long)

Copied to clipboard

Zhiyuan Yao, Zishan Xu, Yifu Guo, Zhiguang Han, Cheng Yang, Shuo Zhang, Weinan Zhang, Xingshan Zeng, Weiwen Liu

Challenge:	Existing routers that use hardcoded tools are limited by scalability and generality bottlenecks.
Approach:	They propose a pipeline for training history-aware routers to empower precise navigation in large-scale ecosystems.
Outcome:	The proposed pipeline can train routers with dynamic context understanding to create the plug-and-play Light Routing Agent.

ProSparse: Introducing and Enhancing Intrinsic Activation Sparsity within Large Language Models (2025.coling-main)

Copied to clipboard

Chenyang Song, Xu Han, Zhengyan Zhang, Shengding Hu, Xiyu Shi, Kuai Li, Chen Chen, Zhiyuan Liu, Guangli Li, Tao Yang, Maosong Sun

DocRED: A Large-Scale Document-Level Relation Extraction Dataset (P19-1)

Challenge:	Activation sparsity is a promising paradigm for accelerating model inference . few large language models achieve high activation spar and comparable performance .
Approach:	They propose a method to achieve activation sparsity and acceleration in large language models . they introduce ReLU activation and adopt progressive sparse regularization .
Outcome:	The proposed method achieves high activation sparsity and comparable model performance.

Copied to clipboard

Yuan Yao, Deming Ye, Peng Li, Xu Han, Yankai Lin, Zhenghao Liu, Zhiyuan Liu, Lixin Huang, Jie Zhou, Maosong Sun