Papers by Pengfei Xu
Event Ontology Completion with Hierarchical Structure Evolution Networks (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for event detection require predefined schemas, but manual defining is expensive and labor-intensive. |
| Approach: | They propose a task to achieve event clustering, hierarchy expansion and type naming . they propose 'neighbor Contrastive Clustering' module and a Hierarchy-Aware Linking module . |
| Outcome: | The proposed method outperforms baseline methods on three datasets. |
Red-Teaming LLM Multi-Agent Systems via Communication Attacks (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Model-based Multi-Agent Systems (LLM-MAS) have revolutionized complex problem-solving capability by enabling agent collaboration through message-based communications. |
| Approach: | They propose an attack that exploits communication mechanisms in Large Language Model-based Multi-Agent Systems (LLM-MAS) by intercepting and manipulating inter-agent messages. |
| Outcome: | The proposed attack exploits communication mechanisms in large language model-based multi-agent systems by intercepting and manipulating inter-agencies. |
Tug-of-War between Knowledge: Exploring and Resolving Knowledge Conflicts in Retrieval-Augmented Language Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing knowledge conflicts in RALMs can ensnare them in a tug-of-war between knowledge and evidence, limiting their practical applicability. |
| Approach: | They propose a method called Conflict-Disentangle Contrastive Decoding (CD2) to better calibrate the model’s confidence. |
| Outcome: | The proposed method can resolve knowledge conflicts in large language models with the help of conflict-disentangle contrast decoding (CD2) . |
VALU: A Benchmark for Video Anomaly Temporal Localization and Understanding at Multiple Semantic Levels (2026.acl-long)
Copied to clipboard
Yixiao He, Menghao Zhang, Haifeng Sun, Jing Wang, Kangheng Lin, Jinghan Wang, Chenye Xu, Pengfei Ren, Qi Qi, Jingyu Wang
| Challenge: | Recent advances in Video Large Language Models (Video-LLMs) enhance the ability of VAU models to describe and interpret anomalies. |
| Approach: | They propose a benchmark that explicitly defines anomalies across five semantic levels and provides detailed temporal boundaries and detailed textual descriptions for each. |
| Outcome: | The proposed benchmark defines anomalies across five semantic levels and provides detailed descriptions for each. |
Cutting Off the Head Ends the Conflict: A Mechanism for Interpreting and Mitigating Knowledge Conflicts in Language Models (2024.findings-acl)
Copied to clipboard
Zhuoran Jin, Pengfei Cao, Hongbang Yuan, Yubo Chen, Jiexin Xu, Huaijun Li, Xiaojian Jiang, Kang Liu, Jun Zhao
| Challenge: | Existing methods to expand internal memory boundaries of language models by providing external context can often conflict, leading to knowledge conflicts. |
| Approach: | They propose a method that prunes conflicting attention heads without updating model parameters. |
| Outcome: | The proposed method can flexibly control eight LMs to use internal memory or external context without updating model parameters. |
DemMA: Dementia Multi-Turn Dialogue Agent with Expert-Guided Reasoning and Action Simulation (2026.findings-acl)
Copied to clipboard
Yutong Song, Jiang Wu, Kazi Shaharair Sharif, Pengfei Zhang, Wenjun Huang, Honghui Xu, Nikil Dutt, Amir M. Rahmani
| Challenge: | Simulating dementia patients with large language models is challenging due to the need to model cognitive impairment, emotional dynamics, and nonverbal behaviors over long conversations. |
| Approach: | They propose an expert-guided dementia dialogue agent for multi-turn patient simulation . they introduce a framework that trains a single LLM to jointly generate reasoning traces, patient utterances, and aligned behavioral actions . |
| Outcome: | The proposed model outperforms baselines in persona fidelity, clinical validity, and educational effectiveness. |
Can Multiple Responses from an LLM Reveal the Sources of Its Uncertainty? (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models can produce unreliable or misleading outputs, posing challenges for real-world applications. |
| Approach: | They employ an auxiliary LLM to analyze the patterns of disagreement among LLMs . they validate their framework on AmbigQA, OpenBookQA, and MMLU-Pro . |
| Outcome: | The proposed model can be used to diagnose uncertainty sources in a model with an auxiliary model. |
LINKED: Eliciting, Filtering and Integrating Knowledge in Large Language Model for Commonsense Reasoning (2024.findings-emnlp)
Copied to clipboard
Jiachun Li, Pengfei Cao, Chenhao Wang, Zhuoran Jin, Yubo Chen, Kang Liu, Xiaojian Jiang, Jiexin Xu, Jun Zhao
| Challenge: | Large language models (LLMs) often exhibit poor performance on knowledge-intensive tasks, such as commonsense reasoning. |
| Approach: | They propose a method to elicit, filter and integrate knowledge in large language models (LINKED) they propose 'reward model' to filter out noisy knowledge and 'take marginal consistent reasoning module' |
| Outcome: | The proposed method outperforms SOTA baselines on two commonsense reasoning tasks. |
Learning How and What to Memorize: Cognition-Inspired Two-Stage Optimization for Evolving Memory (2026.acl-long)
Copied to clipboard
Derong Xu, Shuochen Liu, Pengfei Luo, Pengyue Jia, Yingyi Zhang, Yi Wen, Yimin Deng, Wenlin Zhang, Enhong Chen, Xiangyu Zhao, Tong Xu
| Challenge: | Existing memory systems rely on static, hand-crafted update rules for personalization, but sparse outcome rewards provide weak supervision, resulting in unstable long-horizon optimization. |
| Approach: | They propose a memory guideline optimization framework that learns how memory should be organized and what information to update. |
| Outcome: | The proposed framework learns how memory should be organized and what information to update. |
EvoNarrator: Modeling Scientific Evolution for Feasible Hypothesis Generation (2026.acl-long)
Copied to clipboard
| Challenge: | Scientific discovery evolution does not occur ex nihilo but is characterized by structural deepening and reconfiguration of existing functionalities. |
| Approach: | They propose a framework for hypothesis generation based on evolutionary narratives . they extract structured P-M-L-F quadruples from citation networks and introduce a mechanism to assess their semantic compatibility. |
| Outcome: | The proposed framework reduces logical disconnects by evaluating its semantic compatibility. |
FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluation (D18-1)
Copied to clipboard
| Challenge: | Empirical results show that even the most competitive few-shot learning models struggle on this task, especially as compared with humans. |
| Approach: | They propose a Few-Shot Relation Classification Dataset consisting of 70, 000 sentences on 100 relations derived from Wikipedia and annotated by crowdworkers. |
| Outcome: | The proposed methods perform well on the most competitive few-shot learning models, especially as compared with humans. |
Exploring Memorization in Fine-tuned Language Models (2024.acl-long)
Copied to clipboard
Shenglai Zeng, Yaxin Li, Jie Ren, Yiding Liu, Han Xu, Pengfei He, Yue Xing, Shuaiqiang Wang, Jiliang Tang, Dawei Yin
| Challenge: | Existing studies have shown that pre-trained langauge models tend to memorize and regenerate segments of their pre-training corpus when prompted appropriately. |
| Approach: | They conduct the first comprehensive analysis to explore language models’ memorization during fine-tuning across tasks. |
| Outcome: | The proposed analysis shows that memorization presents a strong disparity among different fine-tuning tasks. |
The Good and The Bad: Exploring Privacy Issues in Retrieval-Augmented Generation (RAG) (2024.findings-acl)
Copied to clipboard
Shenglai Zeng, Jiankun Zhang, Pengfei He, Yiding Liu, Yue Xing, Han Xu, Jie Ren, Yi Chang, Shuaiqiang Wang, Dawei Yin, Jiliang Tang
| Challenge: | Retrieval-augmented generation (RAG) is a powerful technique to facilitate language model generation with proprietary and private data, where data privacy is . a privacy issue that is currently under-explored, is posed by RAG. |
| Approach: | They propose to use retrieval-augmented generation (RAG) to facilitate language model generation with proprietary and private data where data privacy is a pivotal concern. |
| Outcome: | The proposed attack methods demonstrate that RAG can mitigate the old risks, i.e., leakage of the LLMs’ training data. |
Data Poisoning for In-context Learning (2025.findings-naacl)
Copied to clipboard
| Challenge: | In-context learning (ICL) has emerged as a capability of large language models (LLMs) but there is limited understanding of its vulnerability against data poisoning attacks. |
| Approach: | They propose an attack method that exploits ICL’s unique learning mechanisms by identifying discrete text perturbations that influence LLM hidden states. |
| Outcome: | The proposed attack method exploits ICL’s learning mechanisms by identifying discrete text perturbations that influence LLM hidden states. |
Knowledge-Centric Hallucination Detection (2024.emnlp-main)
Copied to clipboard
Xiangkun Hu, Dongyu Ru, Lin Qiu, Qipeng Guo, Tianhang Zhang, Yang Xu, Yun Luo, Pengfei Liu, Yue Zhang, Zheng Zhang
| Challenge: | Large Language Models (LLMs) have shown impressive capabilities but a tendency to hallucinate. |
| Approach: | They propose a framework that introduces claim-triplets to represent claims in LLM responses and evaluates them against a reference. |
| Outcome: | The proposed framework outperforms prior methods by 18.2 to 27.2 points on a benchmark spanning various NLP tasks and annotated 11k claim-triplets from 2.1k responses by seven LLMs. |
RICO: Improving Accuracy and Completeness in Image Recaptioning via Visual Reconstruction (2025.emnlp-main)
Copied to clipboard
Yuchi Wang, Yishuo Cai, Shuhuai Ren, Sihan Yang, Linli Yao, Yuanxin Liu, Yuanxing Zhang, Pengfei Wan, Xu Sun
| Challenge: | Existing recaptioning methods suffer from inaccuracies due to missing fine-grained details. |
| Approach: | They propose a framework that refines captions through visual reconstruction using a text-to-image model and a visual reconstruction framework. |
| Outcome: | The proposed framework outperforms baselines on CapsBench and CompreCap by 10%. |
KG-RAG: Enhancing GUI Agent Decision-Making via Knowledge Graph-Driven Retrieval-Augmented Generation (2025.emnlp-main)
Copied to clipboard
Ziyi Guan, Jason Chun Lok Li, Zhijian Hou, Pingping Zhang, Donglai Xu, Yuzhi Zhao, Mengyang Wu, Jinpeng Chen, Thanh-Toan Nguyen, Pengfei Xian, Wenao Ma, Shengchao Qin, Graziano Chesi, Ngai Wong
| Challenge: | Recent advances in GUI agents have limited app-specific knowledge of complex mobile tasks. |
| Approach: | They propose a Knowledge Graph-driven Retrieval-Augmented Generation framework that transforms fragmented UTGs into structured vector databases for efficient real-time retrieval. |
| Outcome: | The proposed framework outperforms existing methods in a 75.8% success rate and 84.6% decision accuracy test across mobile apps. |
OpenResearcher: Unleashing AI for Accelerated Scientific Research (2024.emnlp-demo)
Copied to clipboard
Yuxiang Zheng, Shichao Sun, Lin Qiu, Dongyu Ru, Cheng Jiayang, Xuefeng Li, Jifan Lin, Binjie Wang, Yun Luo, Renjie Pan, Yang Xu, Qingkai Min, Zizhao Zhang, Yiwen Wang, Wenjie Li, Pengfei Liu
| Challenge: | Global scientific publications are growing annually by about 4%-5% (Pinedo et al., 2024). |
| Approach: | They introduce an AI-assisted platform that answers diverse questions from researchers using Retrieval-Augmented Generation (RAG) they develop various tools to understand queries, search from the scientific literature, filter retrieved information, provide accurate and comprehensive answers, and self-refine answers. |
| Outcome: | OpenResearcher is built on Retrieval-Augmented Generation (RAG) to integrate Large Language Models (LLMs) with up-to-date, domain-specific knowledge. |
On the Generalization of Training-based ChatGPT Detection Methods (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies show that training-based methods are ineffective to detect LLM generated texts from unseen tasks or topics which are not collected during training. |
| Approach: | They propose to train classification models to distinguish LLMs from human texts by a distribution shift caused by prompts, text lengths, topics, and language tasks. |
| Outcome: | The proposed methods can detect LLMs from black-box models, but they suffer from distribution shifts due to a wide range of factors, including prompts, text lengths, topics, and language tasks. |
SegTune: Structured and Fine-Grained Control for Song Generation (2026.acl-long)
Copied to clipboard
Yuejiao Wang, Zihao Ji, Pengfei Cai, Xu Li, Haorui Zheng, Zewen Song, Zhongliang Liu, Chen Zhang, Pengfei Wan
| Challenge: | Recent advances in neural song generation have enabled high-quality synthesis from lyrics and global textual prompts. |
| Approach: | They propose a framework that allows users to specify local musical descriptions aligned to song segments. |
| Outcome: | The proposed framework outperforms baselines in musicality and controllability. |
Towards Understanding Jailbreak Attacks in LLMs: A Representation Space Analysis (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) are susceptible to a type of attack known as jailbreaking, which misleads LLMs to output harmful contents. |
| Approach: | They propose to leverage hidden representations into existing jailbreak targets to move the attacks along the acceptance direction. |
| Outcome: | The proposed methods are validated using the objective of existing jailbreak attacks. |
Towards Better Chain-of-Thought: A Reflection on Effectiveness and Faithfulness (2025.findings-acl)
Copied to clipboard
| Challenge: | Chain-of-thought (CoT) prompting demonstrates varying performance under different reasoning tasks. |
| Approach: | They propose to recall extra information from the question to enhance CoT generation and evaluate CoTs based on their information gain. |
| Outcome: | The proposed method improves both the faithfulness and effectiveness of CoT and evaluates it based on their information gain. |
Towards Dynamic Theory of Mind: Evaluating LLM Adaptation to Temporal Evolution of Human States (2025.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks assess basic Theory of Mind abilities but neglect temporal evolution of mental states in real-world social contexts. |
| Approach: | They propose a benchmark specifically designed to evaluate Large Language Models' ability to understand and track the temporal progression of mental states across interconnected scenarios. |
| Outcome: | The proposed benchmarks underperform humans by 44.7% and show that they can model the dynamic nature of human mental states better than existing models. |
Mitigating the Privacy Issues in Retrieval-Augmented Generation (RAG) via Pure Synthetic Data (2025.emnlp-main)
Copied to clipboard
Shenglai Zeng, Jiankun Zhang, Pengfei He, Jie Ren, Tianqi Zheng, Hanqing Lu, Han Xu, Hui Liu, Yue Xing, Jiliang Tang
| Challenge: | Existing literature suggests that RAG systems may face privacy issues when the retrieval process involves private data. |
| Approach: | They propose a two-stage synthetic data generation paradigm that uses attributes to preserve contextual information from the original data. |
| Outcome: | The proposed approach preserves key contextual information from the original data while reducing privacy risks. |
Complex Event Schema Induction with Knowledge-Enriched Diffusion Model (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies on event schema induction have been hindered by errors and data quality issues. |
| Approach: | They propose a knowledge-enriched discrete diffusion model that distills event scenario knowledge from LLMs. |
| Outcome: | The proposed model achieves outstanding performance across evaluation metrics. |
Ambiguity Awareness Optimization: Towards Semantic Disambiguation for Direct Preference Optimization (2025.emnlp-main)
Copied to clipboard
| Challenge: | Direct Preference Optimization (DPO) is a widely used reinforcement learning from human feedback (RLHF) method across various domains. |
| Approach: | They propose an approach that automatically re-weights ambiguous content to reduce ambiguities by calculating semantic similarity from preference pairs. |
| Outcome: | The proposed approach outperforms state-of-the-art approaches in performance across multiple model scales and widely adopted benchmark datasets. |
RAG-RewardBench: Benchmarking Reward Models in Retrieval Augmented Generation for Preference Alignment (2025.findings-acl)
Copied to clipboard
Zhuoran Jin, Hongbang Yuan, Tianyi Men, Pengfei Cao, Yubo Chen, Jiexin Xu, Huaijun Li, Xiaojian Jiang, Kang Liu, Jun Zhao
| Challenge: | Existing retrieval augmented language models often overlook effective alignment with human preferences. |
| Approach: | They propose a benchmark to evaluate RMs in retrieval augmented language models . they incorporate 18 RAG subsets, six retrievers, and 24 RALMs to increase diversity . |
| Outcome: | The proposed benchmark combines 18 RAG subsets, six retrievers, and 24 RALMs to increase diversity of data sources. |
Hierarchical Relation Extraction with Coarse-to-Fine Grained Attention (D18-1)
Copied to clipboard
| Challenge: | Existing methods for relation extraction use knowledge graphs to automatically label training data . but, it suffers from the wrong labeling problem because not all sentences containing two entities can express their relations in KGs . |
| Approach: | They propose a distant supervision approach to automatically label training instances . they integrate hierarchical information of relations into distantly supervised relation extraction . |
| Outcome: | The proposed model outperforms baseline models on a large-scale dataset. |
Self-supervised Preference Optimization: Enhance Your Language Model with Preference Degree Awareness (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies have focused on replacing the reward model in Reinforcement Learning with Human Feedback (RLHF) methods for Large Language Models (LLMs). |
| Approach: | They propose a self-supervised preference optimization framework that replaces the reward model with a preference loss and alignment loss to improve LLMs' ability to understand human preferences. |
| Outcome: | The proposed framework can be integrated with existing preference optimization methods and significantly boost their performance. |
AgencyBench: Benchmarking the Frontiers of Autonomous Agents in 1M-Token Real-World Contexts (2026.acl-long)
Copied to clipboard
Keyu Li, Junhao Shi, Yang Xiao, Mohan Jiang, Jie Sun, Yunze Wu, Dayuan Fu, Shijie Xia, Xiaojie Cai, Tianze Xu, Weiye Si, Wenjie Li, Dequan Wang, Pengfei Liu
| Challenge: | Existing benchmarks focus on single agentic capability, failing to capture long-horizon real-world scenarios. |
| Approach: | They propose a benchmark that evaluates 6 agentic capabilities across 32 real-world scenarios. |
| Outcome: | Experiments show that closed-source models outperform open-source model (48.4% vs 32.1%) integrating models with advanced scaffolds to form autonomous agents is a paradigm shift. |
AIMMerging: Adaptive Iterative Model Merging Using Training Trajectories for Language Model Continual Learning (2025.emnlp-main)
Copied to clipboard
Yujie Feng, Jian Li, Xiaoyu Dong, Pengfei Xu, Xiaohui Zhou, Yujia Zhang, Zexin Lu, Yasha Wang, Alan Zhao, Xu Chu, Xiao-Ming Wu
| Challenge: | Recent model merging-based methods struggle to effectively manage the trade-off between learning new knowledge and preventing catastrophic forgetting. |
| Approach: | They propose a model merging framework that utilizes learning and forgetting signals from the training trajectory to dynamically monitor the model’s training status. |
| Outcome: | The proposed framework achieves significant performance improvements over existing state-of-the-art methods on three CL benchmarks with various model sizes (from 770M to 13B). |
AgentsCourt: Building Judicial Decision-Making Agents with Court Debate Simulation and Legal Knowledge Augmentation (2024.findings-emnlp)
Copied to clipboard
Zhitao He, Pengfei Cao, Chenhao Wang, Zhuoran Jin, Yubo Chen, Jiexin Xu, Huaijun Li, Kang Liu, Jun Zhao
| Challenge: | Recent advances in deep learning have significantly impacted the legal domain. |
| Approach: | They propose a multi-agent framework for judicial decision-making that simulates the court trial process . they propose 420 Chinese judgment documents to support their framework and build a large-scale legal knowledge base . |
| Outcome: | The proposed framework outperforms existing methods in various aspects, especially in generating legal articles. |
Leros: Learning Explicit Reasoning on Synthesized Data for Commonsense Question Answering (2024.lrec-main)
Copied to clipboard
Chenhao Wang, Pengfei Cao, Jiachun Li, Yubo Chen, Kang Liu, Xiaojian Jiang, Jiexin Xu, Li Qiuxia, Jun Zhao
| Challenge: | Recent work shows large language models can generate useful rationales for commonsense question answering (CQA) however, the cost of deployment and further tuning is relatively expensive for the large models. |
| Approach: | They propose a framework that leverages both knowledge graphs and large language models to synthesize rationale-augmented CQA data. |
| Outcome: | The proposed model can generate useful rationales on unseen CQA benchmarks. |
RLShield: Dynamic Jailbreak Detection for LLMs via Reinforced Adaptive Learning (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to detect jailbreak prompts rely on static model components or fixed decision thresholds. |
| Approach: | They propose a dynamic jailbreak detection framework that employs reinforcement learning for adaptive threshold selection. |
| Outcome: | Experimental results show that the framework outperforms baselines in detection performance while maintaining high computational efficiency. |