Papers by Haojie Zhou
Unveiling and Consulting Core Experts in Retrieval-Augmented MoE-based LLMs (2024.emnlp-main)
Copied to clipboard
Xin Zhou, Ping Nie, Yiwen Guo, Haojie Wei, Zhanqiu Zhang, Pasquale Minervini, Ruotian Ma, Tao Gui, Qi Zhang, Xuanjing Huang
| Challenge: | Existing research seeks to enhance RAG performance by retrieving higher-quality documents or designing RAG-specific LLMs, but internal mechanisms that contribute to RAG’s effectiveness remain underexplored. |
| Approach: | They propose to examine the internal mechanisms within the popular Mixture-of-Expert (MoE)-based LLMs and examine their ability to improve RAG by examining expert activations. |
| Outcome: | The proposed method significantly improved the ability of Large Language Models (LLMs) to solve knowledge-intensive tasks. |
Self-Guided Alignment: Adaptive Preference Sensing for Multi-Objective Generation (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to align LLMs with diverse human values rely on ground-truth scores . existing approaches implicitly approximate an average-user preference, thereby failing to capture heterogeneity of human values or accommodate conflicting user needs. |
| Approach: | They propose a framework that transforms passive reward dependency into an intrinsic adaptive sensing capability. |
| Outcome: | The proposed framework outperforms state-of-the-art models in multiple model scales and improves preference alignment. |