Papers by Pengjie Wang
Retrieval Models Aren’t Tool-Savvy: Benchmarking Tool Retrieval for Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) suffer from inherent inabilities to interact with the physical world and access vast, up-to-date knowledge. |
| Approach: | They propose a tool retrieval benchmark for large language models (LLMs) that includes 7.6k diverse retrieval tasks and a corpus of 43k tools. |
| Outcome: | The proposed model performs poorly on the heterogeneous tool retrieval benchmark, resulting in low pass rate and low retrieval quality. |
Bridging the Capability Gap: Joint Alignment Tuning for Harmonizing LLM-based Multi-Agent Systems (2025.findings-emnlp)
Copied to clipboard
Minghang Zhu, Zhengliang Shi, Zhiwei Xu, Shiguang Wu, Lingjie Wang, Pengjie Ren, Zhaochun Ren, Zhumin Chen
| Challenge: | Existing methods for fine-tuning agents are often inadequate . a multi-agent system can solve complex tasks by dividing responsibilities among specialized agents . |
| Approach: | a new framework is proposed to improve agents collaboration through iterative alignment. |
| Outcome: | The proposed framework outperforms state-of-the-art benchmarks on held-in and held-out tasks. |
SLARD: A Chinese Superior Legal Article Retrieval Dataset (2025.coling-main)
Copied to clipboard
| Challenge: | Existing retrieval methods struggle to achieve ideal results, a study finds . existing large language models lack prior knowledge of the content of superior legal articles . |
| Approach: | They propose to use a Chinese superior legal article retrieval dataset to find relevant articles with higher legal effectiveness. |
| Outcome: | The proposed dataset shows that existing retrieval methods struggle to achieve ideal results. |
KELE: Residual Knowledge Erasure for Enhanced Multi-hop Reasoning in Knowledge Editing (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing knowledge editing techniques show limitations when applied to multi-hop reasoning . residual single-hop knowledge causes edited models to revert to original answers . |
| Approach: | They propose a knowledge editing method that incorporates a Knowledge Erasure mechanism for Large language model Editing (KELE) they propose an erasure function for residual knowledge and an injection function for new knowledge . |
| Outcome: | The proposed method significantly improves multi-hop reasoning capability of edited models. |
Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents (2023.emnlp-main)
Copied to clipboard
Weiwei Sun, Lingyong Yan, Xinyu Ma, Shuaiqiang Wang, Pengjie Ren, Zhumin Chen, Dawei Yin, Zhaochun Ren
| Challenge: | Existing work utilizes generative LLMs for Information Retrieval (IR) rather than direct passage ranking. |
| Approach: | They investigate generative LLMs such as ChatGPT and GPT-4 for relevance ranking in IR and use a test set to verify the model’s ability to rank unknown knowledge. |
| Outcome: | The proposed model outperforms a 3B supervised model on the BEIR benchmark. |
LoRE: Enhancing Search Relevance with Progressive Chain-of-Thought and Preference Alignment (2026.findings-acl)
Copied to clipboard
Chenji Lu, Zhuo Chen, Hui Zhao, Zhiyuan Zeng, Gang Zhao, Junjie Ren, null Lihaoran, Songyan Liu, Pengjie Wang, Chuan Yu, Jian Xu, Bo Zheng
| Challenge: | E-commerce search relevance is a critical component of retrieval systems. |
| Approach: | They propose a large-generative model for search relevance that trains reasoning knowledge, multi-modal understanding and rule awareness into three core competencies. |
| Outcome: | The proposed model outperforms GPT-5 in Macro-F1 and achieves 27% online gain. |
UIPE: Enhancing LLM Unlearning by Removing Knowledge Related to Forgetting Targets (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to unlearning large language models focus on forgetting target data while overlooking the impact of logically related knowledge on the effectiveness of unlearning. |
| Approach: | They propose a method that removes knowledge highly correlated with the forgetting targets and a technique that remove logically related knowledge from the model. |
| Outcome: | The proposed method significantly improves the performance of the proposed method on the TOFU and WMDP benchmarks. |
Multi-Defendant Legal Judgment Prediction via Hierarchical Reasoning (2023.findings-emnlp)
Copied to clipboard
Yougang Lyu, Jitai Hao, Zihan Wang, Kai Zhao, Shen Gao, Pengjie Ren, Zhumin Chen, Fang Wang, Zhaochun Ren
| Challenge: | Existing methods for predicting judgment results for multiple defendants are ineffective. |
| Approach: | They propose a method to predict the judgment results for each defendant in multi-defendant cases . they formalize the multi-diffendant judgment process as hierarchical reasoning chains . |
| Outcome: | The proposed method can predict the judgment results for multiple defendants in multi-defendant cases. |
Generalizing Few-Shot Named Entity Recognizers to Unseen Domains with Type-Related Features (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Few-shot named entity recognition methods struggle with out-of-domain (OOD) examples due to their reliance on manual labeling for the target domain. |
| Approach: | They propose a framework to enable generalization to an unseen target domain with only a few labeled examples. |
| Outcome: | The proposed framework achieves significant performance improvements on in-domain and cross-domain datasets. |
Autonomous Workflow for Multimodal Fine-Grained Training Assistants Towards Mixed Reality (2024.findings-acl)
Copied to clipboard
Jiahuan Pei, Irene Viola, Haochen Huang, Junxiao Wang, Moonisa Ahsan, Fanghua Ye, Jiang Yiming, Yao Sai, Di Wang, Zhumin Chen, Pengjie Ren, Pablo Cesar
| Challenge: | a fine-grained, comprehensive understanding of multimodal environments remains under-explored. |
| Approach: | They propose an automated workflow for integrating AI agents into extended reality (XR) they propose a cerebral language agent that integrates LLM with memory, planning, and interaction with XR tools and a vision-language agent . |
| Outcome: | The proposed workflow integrates AI agents seamlessly into extended reality (XR) applications for fine-grained training. |
KnowTuning: Knowledge-aware Fine-tuning for Large Language Models (2024.emnlp-main)
Copied to clipboard
Yougang Lyu, Lingyong Yan, Shuaiqiang Wang, Haibo Shi, Dawei Yin, Pengjie Ren, Zhumin Chen, Maarten Rijke, Zhaochun Ren
| Challenge: | Large language models (LLMs) are a default solution for many natural language processing tasks. |
| Approach: | They propose a knowledge-aware fine-tuning method to improve LLMs' knowledge awareness . they propose augmentation and comparison stages to improve accuracy and reliability . |
| Outcome: | The proposed method generates more facts with less factual error rate under fine-grained facts evaluation. |
Trustworthy Medical Question Answering: An Evaluation-Centric Survey (2025.emnlp-main)
Copied to clipboard
Yinuo Wang, Baiyang Wang, Robert Mercer, Frank Rudzicz, Sudipta Singha Roy, Pengjie Ren, Zhumin Chen, Xindi Wang
| Challenge: | achieving comprehensive trustworthiness in medical QA poses significant challenges due to complexity of healthcare data, critical nature of clinical scenarios, and multifaceted dimensions of trustworthy AI. |
| Approach: | They examine six key dimensions of trustworthiness in medical QA . they compare how each dimension is evaluated in existing LLM-based systems . |
| Outcome: | The findings show that large language models have improved patient safety and effectiveness . the models exhibit critical trust failures when deployed in clinical settings . |
CodeV: Issue Resolving with Visual Data (2025.findings-acl)
Copied to clipboard
Linhao Zhang, Daoguang Zan, Quanshun Yang, Zhirong Huang, Dong Chen, Bo Shen, Tianyu Liu, Yongshun Gong, Huang Pengjie, Xudong Lu, Guangtai Liang, Lizhen Cui, Qianxiang Wang
| Challenge: | Large Language Models (LLMs) have expanded to more complex repository-level tasks. |
| Approach: | They propose a first approach to leveraging visual data to enhance the issue-resolving capabilities of Large Language Models (LLMs) they demonstrate the effectiveness of CodeV and provide valuable insights into leveraging visualization to resolve GitHub issues. |
| Outcome: | The proposed approach improves the issue-resolving capabilities of Large Language Models (LLMs) by using visual data. |