Papers by Wenyue Zhang
NPHardEval: Dynamic Benchmark on Reasoning Ability of Large Language Models via Complexity Classes (2024.acl-long)
Copied to clipboard
| Challenge: | Complex reasoning ability is one of the most important features of Large Language Models. |
| Approach: | They propose a new benchmark that measures the reasoning ability of Large Language Models . it contains 900 algorithmic questions belonging to the NP-Hard complexity class . |
| Outcome: | The proposed benchmark contains 900 questions belonging to the NP-Hard complexity class and is updated on a monthly basis. |
Disentangling Logic: The Role of Context in Large Language Model Reasoning Capabilities (2025.findings-acl)
Copied to clipboard
Wenyue Hua, Kaijie Zhu, Lingyao Li, Lizhou Fan, Mingyu Jin, Shuhang Lin, Haochen Xue, Zelong Li, Jindong Wang, Yongfeng Zhang
| Challenge: | Using large language models, large language model models can be used to evaluate reasoning abilities in context-rich scenarios. |
| Approach: | They construct datasets for both propositional logic and abductive logic reasoning with four difficulty levels across 12 distinct domains based on Wikipedia categorization and those with purely abstract variables. |
| Outcome: | The proposed model can be used to benchmark LLMs in real-world scenarios, but not in context-rich scenarios. |
Disentangling Memory and Reasoning Ability in Large Language Models (2025.acl-long)
Copied to clipboard
Mingyu Jin, Weidi Luo, Sitao Cheng, Xinyi Wang, Wenyue Hua, Ruixiang Tang, William Yang Wang, Yongfeng Zhang
| Challenge: | Existing LLMs operate as an opaque process without explicit separation between knowledge retrieval and reasoning steps, making the decision-making process unclear and disorganized. |
| Approach: | They propose a language model inference paradigm that decomposes the complex inference process into two distinct and clear actions: (1) memory recall: which retrieves relevant knowledge, and (2) reasoning: which performs reasoning steps based on the recalled knowledge. |
| Outcome: | The proposed paradigm decomposes the inference process into two distinct and clear actions, memory and reason, guiding the model to distinguish between steps that require knowledge retrieval and those that involve reasoning. |
The Impact of Reasoning Step Length on Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Long reasoning steps in LLMs improve reasoning abilities, but the correlation between their effectiveness and the length of reasoning steps remains largely unknown. |
| Approach: | They conducted experiments that expand and compress the rationale reasoning steps within CoT demonstrations while keeping all other factors constant. |
| Outcome: | The results show that lengthening the reasoning steps in prompts significantly enhances LLMs’ reasoning abilities across multiple datasets. |
LeakDojo: Decoding the Leakage Threats of RAG Systems (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing studies have failed to assess RAG leakage risks for large language models . constructing and maintaining highquality RAG knowledge databases has become increasingly costly . |
| Approach: | They propose a framework for controlled evaluation of RAG leakage using query generation and adversarial instructions. |
| Outcome: | The proposed framework compares six existing attacks across fourteen LLMs, four datasets, and diverse RAG systems. |
Public Sentiment Drift Analysis Based on Hierarchical Variational Auto-encoder (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for detecting public sentiment drift are not designed for sentiment drift detection. |
| Approach: | They propose a Hierarchical Variational Auto-Encoder model to learn better distribution representation and a new drift measure to directly evaluate distribution changes between historical and new data. |
| Outcome: | The proposed model performs better than three existing state-of-the-art methods. |
UP5: Unbiased Foundation Model for Fairness-aware Recommendation (2024.eacl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are gaining a foothold in Recommender Systems (RS) but there is growing concern that LLMs perpetuate stereotypes and may result in unfair recommendations. |
| Approach: | They propose a counterfactually-fair-prompt method for LLM-based recommendation that is based on unbiased foundation mOdels. |
| Outcome: | The proposed method achieves better recommendation performance with a high level of fairness on two real-world datasets. |
System 1 + System 2 = Better World: Neural-Symbolic Chain of Logic Reasoning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Current NLP models require more than the ability to learn informative representations from data for logic tasks. |
| Approach: | They propose an architecture that explicitly conducts neural logic reasoning on top of the representation learning models. |
| Outcome: | The proposed architecture improves on the commonsense knowledge graph completion task on a commonsensible task with the two-system architecture. |
BattleAgent: Multi-modal Dynamic Emulation on Historical Battles to Complement Historical Analysis (2024.emnlp-demo)
Copied to clipboard
Shuhang Lin, Wenyue Hua, Lingyao Li, Che-Jui Chang, Lizhou Fan, Jianchao Ji, Hang Hua, Mingyu Jin, Jiebo Luo, Yongfeng Zhang
| Challenge: | Recent advances in large language models have demonstrated impressive reasoning capabilities, indicating their potential to serve as the foundation for agents. |
| Approach: | They propose a detailed emulation system that combines large vision-language model and multi-agent system to emulate dynamic interactions between multiple agents over a period of time. |
| Outcome: | The proposed system combines large vision-language model and multi-agent system to emulate dynamic interactions between agents and their environments over a period of time. |
Exploring Concept Depth: How Large Language Models Acquire Knowledge and Concept at Different Layers? (2025.coling-main)
Copied to clipboard
Mingyu Jin, Qinkai Yu, Jingyuan Huang, Qingcheng Zeng, Zhenting Wang, Wenyue Hua, Haiyan Zhao, Kai Mei, Yanda Meng, Kaize Ding, Fan Yang, Mengnan Du, Yongfeng Zhang
| Challenge: | Large language models have shown remarkable performances across a wide range of tasks, but mechanisms by which they encode tasks of varying complexity remain poorly understood. |
| Approach: | They propose to explore the possibility that LLMs process concepts in different layers . they propose to categorize concepts based on their level of abstraction . |
| Outcome: | The proposed model can process complex concepts in shallow layers, the authors show . the proposed model could be used to prob complex tasks in shallow ones . |
ADO: Automatic Data Optimization for Inputs in LLM Prompts (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent research has focused on refining instruction components and augmenting input data with in-context examples, but this study explores the potential benefits of optimizing the input data itself. |
| Approach: | They propose a content engineering and structural reformulation strategy to optimize input data within prompts to improve performance of Large Language Models. |
| Outcome: | The proposed approach improves performance of Large Language Models (LLMs) in various tasks, offering a promising avenue for future research in prompt engineering. |
AutoSearch: Adaptive Search Depth for Efficient Agentic RAG via Reinforcement Learning (2026.findings-acl)
Copied to clipboard
Jingbo Sun, Wenyue Chong, Songjun Tu, Qichao Zhang, Yaocheng Zhang, Jiajun Chai, Xiaohan Wang, Wei Lin, Guojun Yin, Dongbin Zhao
| Challenge: | Prior work limits search depth to reduce cost, but this often leads to underexploration of complex questions. |
| Approach: | They propose a reinforcement learning framework that evaluates each search step via self-generated intermediate answers. |
| Outcome: | Extensive experiments on multiple benchmarks show that AutoSearch achieves a superior accuracy-efficiency trade-off, alleviating over-searching while preserving search quality. |
TrustAgent: Towards Safe and Trustworthy LLM-based Agents (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing LLMs are primarily used for simple text-related tasks, but LLM-based agents can undertake more complex tasks that require planning and interaction with the physical world and humans. |
| Approach: | They propose an Agent-Constitution-based agent framework with a particular focus on improving the LLM-based agents' safety. |
| Outcome: | The proposed framework can enhance an LLM agent’s safety across multiple domains by identifying and mitigating potential dangers during the planning process. |
EmojiPrompt: Generative Prompt Obfuscation for Privacy-Preserving Communication with Cloud-based LLMs (2025.naacl-long)
Copied to clipboard
| Challenge: | Recent advances in Large Language Models (LLMs) have substantially expanded their applicability across diverse fields, such as personalized recommendations, health report analysis, and financial decision-making. |
| Approach: | They propose a generative transformation paradigm that obfuscates user data with linguistic and non-linguistic elements before submitting it to cloud-based LLMs. |
| Outcome: | The proposed paradigm obfuscates user private data while maintaining performance compared to the unobflated version. |