Papers by Ningyu Zhang

OceanGPT: A Large Language Model for Ocean Science Tasks (2024.acl-long)

Challenge:	Existing Reinforcement Learning approaches rely on outcome-oriented rewards to reinforce fabricated reasoning paths when the final answer is correct.
Approach:	They propose a framework that integrates factual supervision directly into reasoning . they propose to decompose chain of thought into atomic facts and verify them against ground-truth knowledge .
Outcome:	The proposed framework reduces the Incorrect Rate on SimpleQA by 20.3% while maintaining strong performance on complex reasoning benchmarks.

Copied to clipboard

Zhen Bi, Ningyu Zhang, Yida Xue, Yixin Ou, Daxiong Ji, Guozhou Zheng, Huajun Chen

IEPile: Unearthing Large Scale Schema-Conditioned Information Extraction Corpus (2024.acl-short)

Challenge:	Recent advances in Large Language Models (LLMs) have transformed the paradigm in ocean science.
Approach:	They propose a framework to automatically obtain large volume of ocean domain instruction data, which generates instructions based on multi-agent collaboration.
Outcome:	The proposed framework shows a higher level of knowledge expertise for ocean science tasks and gains preliminary embodied intelligence capabilities in ocean technology.

Copied to clipboard

Honghao Gui, Lin Yuan, Hongbin Ye, Ningyu Zhang, Mengshu Sun, Lei Liang, Huajun Chen

LightThinker: Thinking Step-by-Step Compression (2025.emnlp-main)

Challenge:	Large Language Models exhibit a significant performance gap in Information Extraction (IE) high-quality instruction data is the vital key for enhancing LLMs' specific capabilities .
Approach:	They propose a bilingual (English and Chinese) IE instruction corpus that contains 0.32B tokens.
Outcome:	The proposed model improves the performance of LLMs for IE with zero-shot generalization.

Copied to clipboard

Jintian Zhang, Yuqi Zhu, Mengshu Sun, Yujie Luo, Shuofei Qiao, Lun Du, Da Zheng, Huajun Chen, Ningyu Zhang

What Makes AI Research Replicable? Executable Knowledge Graphs as Scientific Knowledge Representations (2026.acl-short)

Challenge:	Recent advances in Large Language Models have demonstrated their remarkable capabilities in complex reasoning tasks, but their efficiency is hindered by the substantial memory and computational costs associated with generating lengthy tokens.
Approach:	They propose a method that dynamically compresses verbose thought steps into compact representations and discards original reasoning chains.
Outcome:	The proposed method reduces peak memory usage and inference time while maintaining competitive accuracy.

Copied to clipboard

Yujie Luo, Zhuoyun Yu, Xuehai Wang, Yuqi Zhu, Ningyu Zhang, Lanning Wei, Lun Du, Da Zheng, Huajun Chen

Generative Knowledge Graph Construction: A Review (2022.emnlp-main)

Challenge:	Existing approaches to replicate AI research are limited by insufficient background knowledge and the limitations of retrieval-augmented generation methods.
Approach:	They propose a pluggable, paper-centric knowledge base that integrates code snippets and technical insights extracted from scientific literature into a verifiable, executable representation.
Outcome:	The proposed knowledge base shows significant performance gains on paperBench when integrated into three agent frameworks with two different LLMs.

Copied to clipboard

Hongbin Ye, Ningyu Zhang, Hui Chen, Huajun Chen

Structured Data in NLP

Challenge:	Knowledge Graphs (KGs) are a form of structured knowledge that rely almost exclusively on human-curated structured or semi-structured data.
Approach:	They propose to use the sequence-to-sequence framework to build knowledge graphs.
Outcome:	The proposed methods have been compared with existing methods and are promising for the future.

MLBiNet: A Cross-Sentence Collective Event Detection Network (2021.acl-long)

Copied to clipboard

Dongfang Lou, Zhilin Liao, Shumin Deng, Ningyu Zhang, Huajun Chen

LightNER: A Lightweight Tuning Paradigm for Low-resource NER via Pluggable Prompting (2022.coling-1)

Challenge:	Detecting multiple events from natural language text is a challenge because of the following problems: a) Sentence-level contextual representation and document-level information aggregation are not enough to detect event triggers.
Approach:	They propose a multi-layer bidirectional network to capture document-level association of events and semantic information simultaneously.
Outcome:	The proposed approach improves performance over the current state-of-the-art approach.

Copied to clipboard

Xiang Chen, Lei Li, Shumin Deng, Chuanqi Tan, Changliang Xu, Fei Huang, Luo Si, Huajun Chen, Ningyu Zhang

Low-Resource NLP

Challenge:	Existing approaches for Named Entity Recognition (NER) use extensive labeled data for model training, which struggles in low-resource scenarios.
Approach:	They propose a lightweight tuning paradigm for low-resource NER via pluggable prompting . they construct a learnable verbalizer of entity categories without any label-specific classifiers .
Outcome:	The proposed model outperforms baselines and class transfer models in low-resource scenarios.

Good Visual Guidance Make A Better Extractor: Hierarchical Visual Prefix for Multimodal Entity and Relation Extraction (2022.findings-naacl)

Copied to clipboard

Xiang Chen, Ningyu Zhang, Lei Li, Yunzhi Yao, Shumin Deng, Chuanqi Tan, Fei Huang, Luo Si, Huajun Chen

Constraining Sequential Model Editing with Editing Anchor Compression (2025.findings-naacl)

Challenge:	Existing approaches for named entity recognition and relation extraction suffer from error sensitivity when irrelevant object images are incorporated in texts.
Approach:	They propose a hierarchical visual prefix fusion NeTwork for visual-enhanced entity and relation extraction using pluggable visual prefixed visual features.
Outcome:	The proposed method achieves state-of-the-art on three benchmark datasets.

Copied to clipboard

Hao-Xiang Xu, Jun-Yu Ma, Zhen-Hua Ling, Ningyu Zhang, Jia-Chen Gu

CKnowEdit: A New Chinese Knowledge Editing Dataset for Linguistics, Facts, and Logic Error Correction in LLMs (2025.acl-long)

Challenge:	Large language models (LLMs) exhibit hallucinations due to incorrect or outdated knowledge embedded in their parameters.
Approach:	They propose a framework to constrain the deviation of the parameter matrix during sequential editing by selecting editing anchors that are important in encoding new relations without deviating too much from the original matrix.
Outcome:	The proposed framework minimizes deviations caused by model editing while retaining over 70% of the general abilities.

Copied to clipboard

Jizhan Fang, Tianhe Lu, Yunzhi Yao, Ziyan Jiang, Xin Xu, Huajun Chen, Ningyu Zhang

Long-tail Relation Extraction via Knowledge Graph Embeddings and Graph Convolution Networks (N19-1)

Challenge:	CKnowEdit is the first-ever knowledge editing dataset designed to correct linguistic, factual, and logical errors in Large Language Models.
Approach:	They propose a Chinese knowledge editing dataset to correct linguistic, factual, and logical errors in Large Language Models.
Outcome:	The proposed dataset highlights the challenges that LLMs face in mastering Chinese . CKnowEdit can correct linguistic, factual, and logical errors in Chinese, the authors show .

Copied to clipboard

Ningyu Zhang, Shumin Deng, Zhanlin Sun, Guanying Wang, Xi Chen, Wei Zhang, Huajun Chen

Exploring Model Kinship for Merging Large Language Models (2025.findings-emnlp)

Challenge:	Existing distance supervised relation extraction models for long-tail data are inadequate for many applications.
Approach:	They propose to leverage implicit relational knowledge among class labels and learn explicit relational knowing using graph convolution networks.
Outcome:	The proposed approach outperforms baselines for long-tail relations on a large-scale dataset.

Copied to clipboard

Yedi Hu, Yunzhi Yao, Ningyu Zhang, Huajun Chen, Shumin Deng

Can We Predict Before Executing Machine Learning Agents? (2026.acl-long)

Challenge:	Model merging has become one of the key technologies for enhancing the capabilities and efficiency of Large Language Models.
Approach:	They propose a model merging strategy that incorporates model kinship to improve model performance.
Outcome:	The proposed model merging strategy can yield better performance on benchmark datasets.

Copied to clipboard

Jingsheng Zheng, Jintian Zhang, Yujie Luo, Yuren Mao, Yunjun Gao, Lun Du, Huajun Chen, Ningyu Zhang

Challenge:	Existing approaches to scientific discovery rely on expensive physical execution . a Generate-Execute-Feedback paradigm is costly and slow .
Approach:	They propose to internalize execution priors to substitute costly runtime checks with instantaneous predictive reasoning, drawing inspiration from World Models.
Outcome:	The proposed framework achieves 61.5% accuracy and robust confidence calibration when primed with a Verified Data Analysis Report.

Novel Relation Detection: Discovering Unknown Relation Types via Multi-Strategy Self-Supervised Learning (2023.findings-emnlp)

Copied to clipboard

Qingbin Liu, Yin Kung, Yanchao Hao, Dianbo Sui, Siyuan Cheng, Xi Chen, Ningyu Zhang, Jiaoyan Chen

Low-Resource NLP

Challenge:	Existing approaches to relation extraction can only recognize predefined relation types . new or out-of-scope relation types may continually emerge after the model is deployed .
Approach:	They propose a novel relation detection task that uses self-supervised learning to handle shallow semantic similarity problem.
Outcome:	The proposed method outperforms state-of-the-art methods on two datasets.

Towards General Agentic Intelligence via Environment Scaling (2026.findings-acl)

Copied to clipboard

Runnan Fang, Shihao Cai, Baixuan Li, Jialong Wu, Guangyu Li, Wenbiao Yin, Xinyu Wang, Xiaobin Wang, Liangcai Su, Zhen Zhang, Shibin Wu, Zhengwei Tao, Yong Jiang, Pengjun Xie, Ningyu Zhang, Fei Huang, Wentao Zhang, Jingren Zhou

Knowledge Rumination for Pre-trained Language Models (2023.emnlp-main)

Challenge:	Diverse real-world APIs require precise, robust function-calling intelligence, which needs agents to develop these capabilities through interaction in varied environments.
Approach:	They propose a framework that scales up environments to enable agentic intelligence . they use a two-phase agent fine-tuning strategy to first endow agents with basic agentic capabilities, then specializing them for domain-specific contexts.
Outcome:	Experiments on -bench, -Bench, and ACEBench show that the model significantly enhances the models’ function-calling capability.

Copied to clipboard

Yunzhi Yao, Peng Wang, Shengyu Mao, Chuanqi Tan, Fei Huang, Huajun Chen, Ningyu Zhang

Bridging Text and Knowledge with Multi-Prototype Embedding for Few-Shot Relational Triple Extraction (2020.coling-main)

Challenge:	Existing studies have shown that pre-trained language models lack the capacity to handle knowledge-intensive tasks alone.
Approach:	They propose a new paradigm to help pre-trained language models utilize latent knowledge without retrieving it from external corpus.
Outcome:	The proposed paradigm can be applied to pre-trained language models without retrieving external knowledge from the corpus.

Copied to clipboard

Haiyang Yu, Ningyu Zhang, Shumin Deng, Hongbin Ye, Wei Zhang, Huajun Chen

Knowledge Editing for Large Language Models (2024.lrec-tutorials)

Challenge:	Existing approaches to supervised relational triple extraction require huge amounts of labeled data.
Approach:	They propose a multi-prototype embedding network model to extract the composition of relational triples from unstructured text.
Outcome:	The proposed method improves the performance of the few-shot relational triple extraction problem.

Copied to clipboard

Ningyu Zhang, Yunzhi Yao, Shumin Deng

AutoAct: Automatic Agent Learning from Scratch for QA via Self-Planning (2024.acl-long)

Challenge:	Large Language Models (LLMs) are not immune to issues of factual accuracy or logically consistent.
Approach:	This tutorial will present cutting-edge methods and practical tools for editing Large Language Models (LLMs).
Outcome:	The aim of this course is to familiarize researchers with the latest advancements and emerging strategies in the realm of knowledge editing for LLMs.

Copied to clipboard

Shuofei Qiao, Ningyu Zhang, Runnan Fang, Yujie Luo, Wangchunshu Zhou, Yuchen Jiang, Chengfei Lv, Huajun Chen

Question Answering

Challenge:	Existing language agent systems struggle with costly data reliance and need multiple models for multiple functions.
Approach:	They propose an automatic agent learning framework for QA that synthesizes planning trajectories without human intervention.
Outcome:	The proposed framework outperforms existing models on question-answering tasks with a division-of-labor strategy.

PRGC: Potential Relation and Global Correspondence Based Joint Relational Triple Extraction (2021.acl-long)

Copied to clipboard

Hengyi Zheng, Rui Wen, Xi Chen, Yifan Yang, Yunyan Zhang, Ziheng Zhang, Ningyu Zhang, Bin Qin, Xu Ming, Yefeng Zheng

Memp: Exploring Agent Procedural Memory (2026.findings-acl)

Challenge:	Recent methods for extracting entities and relations from unstructured texts suffer from limitations, such as redundancy of relation prediction and inefficiency.
Approach:	They propose a joint relational triple extraction framework based on Potential Relation and Global Correspondence (PRGC) they propose overlapping triples for relation prediction and relation-relational alignment .
Outcome:	The proposed framework achieves state-of-the-art performance on public benchmarks with higher efficiency and consistent performance gain on complex scenarios of overlapping triples.

Copied to clipboard

Runnan Fang, Yuan Liang, Xiaobin Wang, Jialong Wu, Shuofei Qiao, Pengjun Xie, Fei Huang, Huajun Chen, Ningyu Zhang

Challenge:	Large Language Models (LLMs) based agents suffer from brittle procedural memory that is manually engineered or entangled in static parameters.
Approach:	They propose a procedural-memory repository that distills past agent trajectories into fine-grained, step-by-step instructions and higher-level, script-like abstractions.
Outcome:	The proposed repository can be used to improve agents' performance on travelplanner and Alfworld.

HyperLoRA: Efficient Cross-task Generalization via Constrained Low-Rank Adapters Generation (2024.findings-emnlp)

Copied to clipboard

Chuancheng Lv, Lei Li, Shitou Zhang, Gang Chen, Fanchao Qi, Ningyu Zhang, Hai-Tao Zheng

Detoxifying Large Language Models via Knowledge Editing (2024.acl-long)

Challenge:	Existing approaches to adapt pre-trained language models (PLMs) to emerging tasks are costly and inefficient.
Approach:	They propose a meta-network that generates task-specific weights without any optimization.
Outcome:	The proposed approach has flexible generalization ability and superior performance over hypenetworks.

Copied to clipboard

Mengru Wang, Ningyu Zhang, Ziwen Xu, Zekun Xi, Shumin Deng, Yunzhi Yao, Qishen Zhang, Linyi Yang, Jindong Wang, Huajun Chen

CaKE: Circuit-aware Editing Enables Generalizable Knowledge Learners (2025.emnlp-main)

Challenge:	Existing methods to detoxify Large Language Models (LLMs) are limiting, but knowledge editing can be effective.
Approach:	They propose a baseline method to detoxify Large Language Models (LLMs) they propose supervised fine-tuning and reinforcement learning from human feedback (RLHF)
Outcome:	The proposed method reduces toxicity of large language models with one instance of tuning . it reduces the toxicity, while minimizing the toxins, the authors show .

Copied to clipboard

Yunzhi Yao, Jizhan Fang, Jia-Chen Gu, Ningyu Zhang, Shumin Deng, Huajun Chen, Nanyun Peng

SPEECH: Structured Prediction with Energy-Based Event-Centric Hyperspheres (2023.acl-long)

Challenge:	Existing knowledge editing methods fail to generalize updates to multi-hop reasoning tasks . Existing methods only edit single or a few model layers, inadequately integrate updated knowledge into reasoning pathways.
Approach:	They propose a circuit-aware method that enhances the effective integration of updated knowledge in large language models by leveraging curated data samples guided by their analysis.
Outcome:	The proposed method improves accuracy and accuracy of 20% on the MQuAKE dataset while requiring less memory.

Copied to clipboard

Shumin Deng, Shengyu Mao, Ningyu Zhang, Bryan Hooi

Multimodality

Challenge:	Experiments on two unified-annotated event datasets indicate that SPEECH is predominant in event detection and event-relation extraction tasks.
Approach:	They propose to model complex dependency among event structured components with energy-based energy-modeling and represent event classes with simple but effective hyperspheres.
Outcome:	Experiments on two unified-annotated event datasets show that SPEECH is predominant in event detection and event-relation extraction tasks.

OntoED: Low-resource Event Detection with Ontology Embedding (2021.acl-long)

Copied to clipboard

Shumin Deng, Ningyu Zhang, Luoqiu Li, Chen Hui, Tou Huaixiao, Mosha Chen, Fei Huang, Huajun Chen

Commonsense Knowledge Salience Evaluation with a Benchmark Dataset in E-commerce (2022.findings-emnlp)

Challenge:	Existing methods to ED rely on training instances and ignore correlation of event types.
Approach:	They propose a process of event ontology population linking event instances to pre-defined event types in event ontoology and ontological embedding to address these problems.
Outcome:	The proposed framework can be applied to new unseen event types by establishing linkages to existing ones.

Copied to clipboard

Yincen Qu, Ningyu Zhang, Hui Chen, Zelin Dai, Chengming Wang, Xiaoyu Wang, Qiang Chen, Huajun Chen

Commonsense Reasoning

Challenge:	Existing models rank statements solely by confidence scores, and there is no information about which ones are salient from a human perspective.
Approach:	They propose a task where a model is required to learn whether a triple is salient . they propose supervised salience evaluation using a new Benchmark dataset .
Outcome:	The proposed task is based on a new Benchmark dataset of salience evaluation in e-commerce . it shows that saliency evaluation is hard, where models perform poorly on evaluation set .

OmniThink: Expanding Knowledge Boundaries in Machine Writing through Thinking (2025.emnlp-main)

Copied to clipboard

Zekun Xi, Wenbiao Yin, Jizhan Fang, Jialong Wu, Runnan Fang, Yong Jiang, Pengjun Xie, Fei Huang, Huajun Chen, Ningyu Zhang

Information Retrieval

Challenge:	Recent advances in Large Language Models (LLMs) have demonstrated remarkable progress in machine writing such as open domain long-form generation.
Approach:	They propose a slow-thinking machine writing framework that emulates the human-like process of iterative expansion and reflection to improve the knowledge density of generated articles.
Outcome:	The proposed framework improves the knowledge density of generated articles without compromising metrics such as coherence and depth.

Reasoning with Language Model Prompting: A Survey (2023.acl-long)

Copied to clipboard

Shuofei Qiao, Yixin Ou, Ningyu Zhang, Xiang Chen, Yunzhi Yao, Shumin Deng, Chuanqi Tan, Fei Huang, Huajun Chen

ReLearn: Unlearning via Learning for Large Language Models (2025.acl-long)

Challenge:	Reasoning is an essential ability for complex problem-solving and can provide back-end support for various real-world applications.
Approach:	They present cutting-edge research on reasoning with language model prompting and provide systematic resources to help beginners.
Outcome:	The proposed approaches have not been systematically reviewed and analyzed.

Copied to clipboard

Haoming Xu, Ningyuan Zhao, Liming Yang, Sendong Zhao, Shumin Deng, Mengru Wang, Bryan Hooi, Nay Oo, Huajun Chen, Ningyu Zhang

Editing Conceptual Knowledge for Large Language Models (2024.findings-emnlp)

Challenge:	Existing methods for unlearning large language models often rely on reverse optimization to reduce target token probabilities.
Approach:	They propose a data augmentation and fine-tuning pipeline for effective unlearning . they propose augmentation, evaluation frameworks to measure contextual forgetting .
Outcome:	The proposed framework achieves targeted forgetting while preserving high-quality outputs.

Copied to clipboard

Xiaohan Wang, Shengyu Mao, Shumin Deng, Yunzhi Yao, Yue Shen, Lei Liang, Jinjie Gu, Huajun Chen, Ningyu Zhang

Summarizing Chinese Medical Answer with Graph Convolution Networks and Question-focused Dual Attention (2020.findings-emnlp)

Challenge:	Existing knowledge editing methods can modify concept-level definitions, but they can distort instantial knowledge in LLMs, leading to poor performance.
Approach:	They construct a benchmark dataset ConceptEdit and establish new metrics for evaluation to investigate the editing capability of LLMs.
Outcome:	The proposed methods can modify concept definitions but can distort instantial knowledge in LLMs, leading to poor performance.

Copied to clipboard

Ningyu Zhang, Shumin Deng, Juan Li, Xi Chen, Wei Zhang, Huajun Chen

Multimodality

Challenge:	Existing approaches to generate answer summarization for medical questions are not straightforward to apply to the medical domain.
Approach:	They propose an approach that utilizes graph convolution networks and question-focused dual attention for Chinese medical answer summarization.
Outcome:	The proposed model generates more coherent and informative summaries compared with baseline models.

DeepKE: A Deep Learning Based Knowledge Extraction Toolkit for Knowledge Base Population (2022.emnlp-demos)

Copied to clipboard

Ningyu Zhang, Xin Xu, Liankuan Tao, Haiyang Yu, Hongbin Ye, Shuofei Qiao, Xin Xie, Xiang Chen, Zhoubo Li, Lei Li

Can We Edit Multimodal Large Language Models? (2023.emnlp-main)

Challenge:	Existing knowledge extraction tools are not complete due to emerging entities and relations in real-world applications.
Approach:	They propose an open-source knowledge extraction toolkit DeepKE that supports low-resource, document-level and multimodal scenarios in the knowledge base population.
Outcome:	The proposed toolkit supports low-resource, document-level and multimodal scenarios in the knowledge base population.

Copied to clipboard

Siyuan Cheng, Bozhong Tian, Qingbin Liu, Xi Chen, Yongheng Wang, Huajun Chen, Ningyu Zhang

Editing Large Language Models: Problems, Methods, and Opportunities (2023.emnlp-main)

Challenge:	Existing methods to edit multimodal models have been used to incrementally infuse a language model with a new set of facts.
Approach:	They construct a benchmark for editing multimodal Large Language Models and establish metrics for evaluation.
Outcome:	The proposed benchmarks show that editing multimodal models is not as difficult as editing single-modal models.

Copied to clipboard

Yunzhi Yao, Peng Wang, Bozhong Tian, Siyuan Cheng, Zhoubo Li, Shumin Deng, Huajun Chen, Ningyu Zhang

Beyond Prompt Engineering: Robust Behavior Control in LLMs via Steering Target Atoms (2025.acl-long)

Challenge:	Recent advances in model editing for LLMs have created challenges and opportunities for the community.
Approach:	They propose to alter the behavior of LLMs efficiently within a specific domain without negatively impacting performance across other inputs.
Outcome:	The proposed method alters behavior of LLMs efficiently within a specific domain without negatively impacting performance across other inputs.

Copied to clipboard

Mengru Wang, Ziwen Xu, Shengyu Mao, Shumin Deng, Zhaopeng Tu, Huajun Chen, Ningyu Zhang

Towards Realistic Low-resource Relation Extraction: A Benchmark with Empirical Baseline Study (2022.findings-emnlp)

Challenge:	Recent research has explored the use of sparse autoencoders (SAE) to disentangle knowledge in high-dimensional spaces for steering.
Approach:	They propose a method that isolates and manipulates disentangled knowledge components to enhance safety by using sparse autoencoders to disentangle knowledge in high-dimensional spaces for steering.
Outcome:	The proposed method is able to isolate and manipulate disentangled knowledge components to enhance safety in large reasoning models.

Copied to clipboard

Xin Xu, Xiang Chen, Ningyu Zhang, Xin Xie, Xi Chen, Huajun Chen

Low-Resource NLP

Challenge:	Existing approaches to extract relational facts from text are limited in their ability to learn from limited labeled data.
Approach:	They propose to use prompt-based methods with few-shot labeled data to evaluate performance . data augmentation technologies and self-training are also proposed to generate more labeles in-domain data.
Outcome:	The proposed methods perform well in low-resource settings with 8 relation extraction datasets.

Finding Influential Instances for Distantly Supervised Relation Extraction (2022.coling-1)

Copied to clipboard

Zifeng Wang, Rui Wen, Xi Chen, Shao-Lun Huang, Ningyu Zhang, Yefeng Zheng

How Do LLMs Acquire New Knowledge? A Knowledge Circuits Perspective on Continual Pre-Training (2025.findings-acl)

Challenge:	Distant supervision models suffer from high label noise and are not reliable for DS.
Approach:	They propose a model-agnostic instance sampling method for relation extraction (RE) by influence function, namely REIF.
Outcome:	The proposed method reduces the computational complexity from O(mn) to O(1), with analyzing its robustness on the selected sampling function.

Copied to clipboard

Yixin Ou, Yunzhi Yao, Ningyu Zhang, Hui Jin, Jiacheng Sun, Shumin Deng, Zhenguo Li, Huajun Chen

Logic-guided Semantic Representation Learning for Zero-Shot Relation Classification (2020.coling-main)

Challenge:	Large Language Models (LLMs) have exceptional capabilities in knowledge-intensive tasks . however, they struggle with knowledge updates due to dynamic nature of world knowledge .
Approach:	They propose to identify computational subgraphs that facilitate knowledge storage and processing . they also identify a phase shift from formation to optimization in LLMs .
Outcome:	The proposed model can capture factual knowledge from pre-training corpus and encapsulate it as extensive parametric knowledge.

Copied to clipboard

Juan Li, Ruoxu Wang, Ningyu Zhang, Wen Zhang, Fan Yang, Huajun Chen

Illusions of Confidence? Diagnosing LLM Truthfulness via Neighborhood Consistency (2026.acl-long)

Challenge:	Existing methods to zero-shot relation classification can only identify seen relations . existing methods rely on descriptive information to improve understandability of relation types .
Approach:	They propose a logic-guided semantic representation learning model for zero-shot relation classification that builds connections between seen and unseen relations via implicit and explicit semantic representations with knowledge graph embeddings and logic rules.
Outcome:	The proposed model can generalize to unseen relation types and achieve promising improvements.

Copied to clipboard

Haoming Xu, Ningyuan Zhao, Yunzhi Yao, Weihong Xu, Hongru Wang, Xinle Deng, Shumin Deng, Jeff Z. Pan, Huajun Chen, Ningyu Zhang

Challenge:	Existing evaluations rely on point-wise confidence, which can mask brittle belief.
Approach:	They propose a measure of belief robustness that evaluates coherence across a conceptual neighborhood.
Outcome:	The proposed model is more resistant to interference than existing models.

Class Lifelong Learning for Intent Detection via Structure Consolidation Networks (2023.findings-acl)

Copied to clipboard

Qingbin Liu, Yanchao Hao, Xiaolong Liu, Bo Li, Dianbo Sui, Shizhu He, Kang Liu, Jun Zhao, Xi Chen, Ningyu Zhang, Jiaoyan Chen

Sentiment Analysis

Challenge:	Existing intent detection models can only handle predefined intent classes in the offline environment.
Approach:	They propose a method that continually learns new intent classes from new data . structure-based retrospection and contrastive knowledge distillation are used to solve these problems .
Outcome:	The proposed method outperforms existing models on three benchmarks.

EasyEdit: An Easy-to-use Knowledge Editing Framework for Large Language Models (2024.acl-demos)

Copied to clipboard

Peng Wang, Ningyu Zhang, Bozhong Tian, Zekun Xi, Yunzhi Yao, Ziwen Xu, Mengru Wang, Shengyu Mao, Xiaohan Wang, Siyuan Cheng, Kangwei Liu, Yuansheng Ni, Guozhou Zheng, Huajun Chen

How Controllable Are Large Language Models? A Unified Evaluation across Behavioral Granularities (2026.acl-long)

Challenge:	Large Language Models (LLMs) suffer from knowledge cutoff or fallacy issues, which means they are unaware of unseen events or generate text with incorrect facts owing to outdated/noisy data.
Approach:	They propose an easy-to-use knowledge editing framework for Large Language Models that allows users to easily edit updated knowledge and adjust undesired behavior while minimizing the impact on unrelated inputs.
Outcome:	The proposed framework surpasses traditional fine-tuning in terms of reliability and generalization.

Copied to clipboard

Ziwen Xu, Kewei Xu, Haoming Xu, Haiwen Hong, Longtao Huang, Hui Xue, Ningyu Zhang, Yongliang Shen, Guozhou Zheng, Huajun Chen, Shumin Deng

Schema-adaptable Knowledge Graph Construction (2023.findings-emnlp)

Challenge:	Large language models (LLMs) are increasingly deployed in socially sensitive domains, yet their unpredictable behaviors pose significant risks.
Approach:	They propose a hierarchical benchmark for evaluating LLM controllability across three domains: language features, sentiment, and personality.
Outcome:	The proposed framework offers a principled and interpretable framework for safe and controllable LLM behavior serving as a foundation for future research.

Copied to clipboard

Hongbin Ye, Honghao Gui, Xin Xu, Xi Chen, Huajun Chen, Ningyu Zhang

Multimodality

Challenge:	Existing Knowledge Graph Construction (KGC) tasks rely on static information extraction with a closed set of pre-defined schemas.
Approach:	They propose a static knowledge Graph Construction task that extracts entity, relation, and event based on dynamically changing schema graph without retraining.
Outcome:	The proposed system outperforms existing methods but still has room for improvement . it can extract entity, relation, and event based on dynamically changing schema graph without re-training .

OneGen: Efficient One-Pass Unified Generation and Retrieval for LLMs (2024.findings-emnlp)

Copied to clipboard

Jintian Zhang, Cheng Peng, Mengshu Sun, Xiang Chen, Lei Liang, Zhiqiang Zhang, Jun Zhou, Huajun Chen, Ningyu Zhang

Green & Sustainable NLP

Challenge:	Recent advances in Large Language Models (LLMs) have significantly enhanced the generative capabilities for various NLP tasks, but they still suffer from hallucinations due to their exclusive reliance on parametric knowledge.
Approach:	They propose a framework that integrates retrieval tokens generated autoregressively into a single LLM to handle both tasks simultaneously in a unified forward pass.
Outcome:	The proposed framework bridges the traditionally separate training approaches for generation and retrieval by incorporating retrieval tokens generated autoregressively.

Noisy-Labeled NER with Confidence Estimation (2021.naacl-main)

Copied to clipboard

Kun Liu, Yao Fu, Chuanqi Tan, Mosha Chen, Ningyu Zhang, Songfang Huang, Sheng Gao

EasyInstruct: An Easy-to-use Instruction Processing Framework for Large Language Models (2024.acl-demos)

Challenge:	Recent studies in deep learning have shown significant progress in named entity recognition (NER) . however, most existing works assume clean data annotation, while real-world data typically involve a large amount of noises.
Approach:	They propose a confidence estimation approach for named entity recognition using noisy labels using local and global independence assumptions.
Outcome:	The proposed method marginalizes out labels of low confidence with a CRF model and integrates it into a self-training framework for boosting performance.

Copied to clipboard

Yixin Ou, Ningyu Zhang, Honghao Gui, Ziwen Xu, Shuofei Qiao, Runnan Fang, Lei Li, Zhen Bi, Guozhou Zheng, Huajun Chen

Making Language Models Better Tool Learners with Execution Feedback (2024.naacl-long)

Challenge:	Large Language Models (LLMs) have improved performance across tasks and domains . instruction tuning is a crucial technique to enhance the capabilities of LLMs - but there is no standard open-source instruction processing framework available for the community .
Approach:	They propose an open-source instruction tuning framework for Large Language Models that modularizes instruction generation, selection, prompting and their combination and interaction.
Outcome:	The proposed framework is open-source and available on Github.

Copied to clipboard

Shuofei Qiao, Honghao Gui, Chengfei Lv, Qianghuai Jia, Huajun Chen, Ningyu Zhang

Knowledge Graph Pooling and Unpooling for Concept Abstraction (2025.coling-main)

Challenge:	Existing tool learning methodologies induce large language models to utilize tools indiscriminately . Existing frameworks that teach language models when and how to use tools propagate errors rather than enhance performance.
Approach:	They propose a framework that enables large language models to continually learn through feedback derived from tool execution.
Outcome:	The proposed framework can make large language models selectively use tools . it improves accuracy while enhancing insufficient tool learning, it shows .

Copied to clipboard

Juan Li, Wen Zhang, Zhiqiang Liu, Mingchen Tu, Mingyang Chen, Ningyu Zhang, Shijian Li

Structured Data in NLP

Challenge:	Knowledge graph embedding (KGE) aims to embed entities and relations as vectors in a continuous space.
Approach:	They propose a framework with KG Pooling and unpooling and Contrastive Learning to abstract and encode latent concepts for better KG prediction.
Outcome:	The proposed framework outperforms baselines on link prediction task.

U-Fold: Dynamic Intent-Aware Context Folding for User-Centric Agents (2026.findings-acl)

Copied to clipboard

Jin Su, Runnan Fang, Yeqiu Li, Xiaobin Wang, Shihao Cai, Pengjun Xie, Ningyu Zhang, Fajie Yuan

Dialogue Systems & Conversational Agents

Challenge:	Existing context-folding methods are designed for single-query or single-intent scenarios.
Approach:	They propose a dynamic context-folding framework tailored to user-centric tasks that preserves fine-grained information through dynamic context folding.
Outcome:	The proposed framework outperforms ReAct and previous folding frameworks on long, noisy tasks.

Field Embedding: A Unified Grain-Based Framework for Word Representation (2021.naacl-main)

Copied to clipboard

Junjie Luo, Xi Chen, Jichao Sun, Yuejia Xiang, Ningyu Zhang, Xiang Wan

Representation Learning

Challenge:	Current methods focus on learning word embeddings while linguistic information is discarded after the learning.
Approach:	They propose a framework field embedding to jointly learn word and grain embedds by incorporating morphological, phonetic, and syntactical linguistic fields.
Outcome:	The proposed framework integrates morphological, phonetic, and syntactical linguistic fields to learn word embeddings and grain embedds.

Attention-Based Capsule Networks with Dynamic Routing for Relation Extraction (D18-1)

Copied to clipboard

Ningyu Zhang, Shumin Deng, Zhanling Sun, Xi Chen, Wei Zhang, Huajun Chen

Are Structural Concepts Universal in Transformer Language Models? Towards Interpretable Cross-Lingual Generalization (2023.findings-emnlp)

Challenge:	Existing neural networks focus on instance representation, and subsampling fails to retain precise spatial relationships between higher-level parts.
Approach:	They propose a neural approach based on capsule networks with attention mechanisms to extract relational information from a capsule.
Outcome:	The proposed method improves the precision of the predicted relations with different benchmarks.

Copied to clipboard

Ningyu Xu, Qi Zhang, Jingting Ye, Menghan Zhang, Xuanjing Huang

OpenUE: An Open Toolkit of Universal Extraction from Text (2020.emnlp-demos)

Challenge:	Large language models (LLMs) have implicitly transfer knowledge across languages, but not all languages have such generalization capabilities.
Approach:	They propose a meta-learning-based method to learn to align conceptual spaces of different languages to enhance cross-lingual generalization.
Outcome:	The proposed method achieves competitive results with state-of-the-art methods and narrows the performance gap between languages.

Copied to clipboard

Ningyu Zhang, Shumin Deng, Zhen Bi, Haiyang Yu, Jiacheng Yang, Mosha Chen, Fei Huang, Wei Zhang, Huajun Chen

Knowledge Mechanisms in Large Language Models: A Survey and Perspective (2024.findings-emnlp)

Challenge:	a large number of natural language processing tasks focus on token-level or sentence-level understandings.
Approach:	They propose an open-source and extensible toolkit for various extraction tasks . they deploy an online demo with restful APIs to support real-time extraction .
Outcome:	The proposed model can be used to extract information from text without training and deployment.

Copied to clipboard

Mengru Wang, Yunzhi Yao, Ziwen Xu, Shuofei Qiao, Shumin Deng, Peng Wang, Xiang Chen, Jia-Chen Gu, Yong Jiang, Pengjun Xie, Fei Huang, Huajun Chen, Ningyu Zhang

EasyEdit2: An Easy-to-use Steering Framework for Editing Large Language Models (2025.emnlp-demos)

Challenge:	Using large language models, we can understand knowledge mechanisms in LLMs for learning, storage, utilization, and evolution.
Approach:	They propose to analyze knowledge mechanisms in Large Language Models (LLMs) they examine utilization, evolution, and the potential dark knowledge (hypothesis) they hope to help understand knowledge in LLMs and provide insights for future research .
Outcome:	The proposed model can be used to analyze the evolution of parametric knowledge in LLMs.

Copied to clipboard

Ziwen Xu, Shuxun Wang, Kewei Xu, Haoming Xu, Mengru Wang, Xinle Deng, Yunzhi Yao, Guozhou Zheng, Huajun Chen, Ningyu Zhang

RaFe: Ranking Feedback Improves Query Rewriting for RAG (2024.findings-emnlp)

Challenge:	Large Language Models (LLMs) have demonstrated extraordinary capabilities, however, they may still generate unreliable or unsafe outputs.
Approach:	They propose a framework that allows plug-and-play adjustability for controlling Large Language Model (LLM) behaviors.
Outcome:	The framework is designed to enable plug-and-play adjustability for controlling Large Language Model (LLM) behaviors.

Copied to clipboard

Shengyu Mao, Yong Jiang, Boli Chen, Xiao Li, Peng Wang, Xinyu Wang, Pengjun Xie, Fei Huang, Huajun Chen, Ningyu Zhang

Text Generation

Challenge:	Large Language Models (LLMs) and Retrieval Augmentation Generation (RAG) techniques have evolved to enhance document retrieval by reformulating queries.
Approach:	They propose a framework for training query rewriting models that leverages a reranker framework.
Outcome:	The proposed framework provides ranking feedback aligned well with the rewriting objectives without needing signals from annotations and supports both online and offline training models.

CBLUE: A Chinese Biomedical Language Understanding Evaluation Benchmark (2022.acl-long)

Copied to clipboard

Ningyu Zhang, Mosha Chen, Zhen Bi, Xiaozhuan Liang, Lei Li, Xin Shang, Kangping Yin, Chuanqi Tan, Jian Xu, Fei Huang, Luo Si, Yuan Ni, Guotong Xie, Zhifang Sui, Baobao Chang, Hui Zong, Zheng Yuan, Linfeng Li, Jun Yan, Hongying Zan, Kunli Zhang, Buzhou Tang, Qingcai Chen

Challenge:	a new benchmark for biomedical language understanding is being developed in Chinese . most benchmarks are limited to English, which makes it difficult to replicate success in other languages.
Approach:	They propose to use Chinese biomedical language understanding evaluation benchmarks to evaluate Chinese models.
Outcome:	The proposed benchmarks show that the current models perform worse than the human ceiling.

Contrastive Demonstration Tuning for Pre-trained Language Models (2022.findings-emnlp)

Copied to clipboard

Xiaozhuan Liang, Ningyu Zhang, Siyuan Cheng, Zhenru Zhang, Chuanqi Tan, Huajun Chen

SynWorld: Virtual Scenario Synthesis for Agentic Action Knowledge Refinement (2025.acl-short)

Challenge:	Recent studies focus on searching discrete or continuous prompts or optimized verbalizers, yet the demonstration examples are crucial for an excellent final performance of prompt-tuning.
Approach:	They propose a pluggable, extensible, and efficient approach to prompt tuning that is free of demonstration sampling.
Outcome:	The proposed approach can be pluggable, extensible, and efficient on 16 datasets.

Copied to clipboard

Runnan Fang, Xiaobin Wang, Yuan Liang, Shuofei Qiao, Jialong Wu, Zekun Xi, Ningyu Zhang, Yong Jiang, Pengjun Xie, Fei Huang, Huajun Chen

Challenge:	Using Large Language Models (LLMs)-based agents can enhance their understanding of environments and tasks.
Approach:	They propose a framework that allows agents to synthesize possible scenarios with multi-step action invocation within the action space and perform Monte Carlo Tree Search exploration to refine their action knowledge in the current environment.
Outcome:	The proposed framework synthesizes possible scenarios with multi-step action invocation within the action space and performs Monte Carlo Tree Search exploration to refine action knowledge in the current environment.

Unified Hallucination Detection for Multimodal Large Language Models (2024.acl-long)

Copied to clipboard

Xiang Chen, Chenxi Wang, Yida Xue, Ningyu Zhang, Xiaoyan Yang, Qiang Li, Yue Shen, Lei Liang, Jinjie Gu, Huajun Chen

Logical Structure as Knowledge: Enhancing LLM Reasoning via Structured Logical Knowledge Density Estimation (2026.findings-acl)

Challenge:	despite significant strides in multimodal tasks, MLLMs are plagued by the critical issue of hallucination.
Approach:	They propose a meta-evaluation benchmark to facilitate evaluation of advancements in hallucination detection methods.
Outcome:	The proposed framework validates hallucinations robustly and provides strategic insights . MHaluBench is a meta-evaluation benchmark designed to facilitate evaluation .

Copied to clipboard

Zhen Bi, Zhenlin Hu, Xueshu Chen, Mingyang Chen, Cheng Deng, Yida Xue, Zhen Wang, Qing Shen, Ningyu Zhang, Jungang Lou

Cross-Linguistic Syntactic Difference in Multilingual BERT: How Good is It and How Does It Affect Transfer? (2022.emnlp-main)

Challenge:	Existing data-centric paradigms equate quality with factuality or diversity and ignore the internal logical complexity of training samples.
Approach:	They propose a density-aware re-cognizing optimization strategy that prioritizes high-density logical samples to align training with the model's reasoning boundary.
Outcome:	The proposed metric outperforms existing methods and improves reasoning performance without increasing total data volume.

Copied to clipboard

Ningyu Xu, Tao Gui, Ruotian Ma, Qi Zhang, Jingting Ye, Menghan Zhang, Xuanjing Huang

KnowAgent: Knowledge-Augmented Planning for LLM-Based Agents (2025.findings-naacl)

Challenge:	Multilingual BERT (mBERT) has demonstrated considerable cross-lingual syntactic ability, but it is not well understood what leads to this variation and whether it fairly reflects difference between languages.
Approach:	They propose to use multilingual BERT to enable zero-shot cross-lingual transfer of syntactic knowledge between different languages by generating grammatical relations in 24 different languages.
Outcome:	The results show that the distance between the distributions of different languages is highly consistent with the syntactic difference in terms of linguistic formalisms.

Copied to clipboard

Yuqi Zhu, Shuofei Qiao, Yixin Ou, Shumin Deng, Shiwei Lyu, Yue Shen, Lei Liang, Jinjie Gu, Huajun Chen, Ningyu Zhang

Why Steering Works: Toward a Unified View of Language Model Parameter Dynamics (2026.acl-long)

Challenge:	Large Language Models (LLMs) fail to effectively guide the planning trajectories during task solving and result in planning hallucinations.
Approach:	They propose a novel approach to enhance the planning capabilities of large language models by incorporating explicit action knowledge.
Outcome:	The proposed approach can achieve comparable or superior performance to existing baselines on HotpotQA and ALFWorld.

Copied to clipboard

Ziwen Xu, Chenyan WU, Hengyu Sun, Haiwen Hong, Mengru Wang, Yunzhi Yao, Longtao Huang, Hui Xue, Shumin Deng, Zhixuan Chu, Huajun Chen, Ningyu Zhang

Efficient and Robust Knowledge Graph Construction (2022.aacl-tutorials)

Challenge:	Methods for controlling large language models (LLMs) are often studied in isolation, obscuring connections and making comparison difficult.
Approach:	They propose a preference-utility analysis that separates control effects into preference and utility, and measures both on a shared log-odds scale using polarity-paired contrastive examples.
Outcome:	The proposed approach improves preference while preserving utility.

Copied to clipboard

Ningyu Zhang, Tao Gui, Guoshun Nan