Papers by Wenjun Peng
From Outcome to Process: Optimizing MoE Load Balancing with MCTS (2026.findings-acl)
Copied to clipboard
Wenjun Ke, Hengyuan Xu, Ziyu Shang, Yao He, Jiahao Wang, Zijie Xu, Peng Wang, Yuhang Lou, Jiajun Liu
| Challenge: | Existing balancing strategies focus on constraining the final distribution of expert usage, but overlook the routing decisions made at each layer. |
| Approach: | They propose a three-stage framework that leverages process-level rewards to guide balanced expert routing. |
| Outcome: | Extensive experiments show that LayerMoE improves the performance of state-of-the-art LoRA-MoA baselines, yielding an average accuracy gain of 1.39%. |
TriEx: A Game-based Tri-View Framework for Explaining Internal Reasoning in Multi-Agent LLMs (2026.acl-long)
Copied to clipboard
| Challenge: | Existing explainability methods for large language models have been limited in capturing interaction-dependent belief dynamics and multi-agent reasoning. |
| Approach: | They propose a tri-view explainability framework that instruments sequential decision making with aligned artifacts. |
| Outcome: | The proposed framework enables analysis of explanation faithfulness, belief dynamics, and evaluator reliability, revealing systematic mismatches between what agents say, what they believe, and what they do. |
On the Consistency of Commonsense in Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing evaluations of commonsense for large language models focus on downstream knowledge tasks, failing to probe whether LLMs truly understand and utilize knowledge or merely memorize it. |
| Approach: | They propose to automatically construct a large benchmark named CoCo which measures LLMs’ knowledge memorization, comprehension, and application and examines the consistency between these tasks. |
| Outcome: | The proposed benchmark systematically assesses LLMs’ knowledge memorization, comprehension, and application and examines the consistency between these tasks. |
Acquisition and Application of Novel Knowledge in Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for constructing new datasets rely on timestamps or simple templates that do not accurately reflect the real world. |
| Approach: | They propose a knowledge dataset construction approach that simulates biological evolution using knowledge graphs to generate synthetic entities with diverse attributes. |
| Outcome: | The proposed framework outperforms knowledge augmentation methods by 3.3%-38%. |
Revisiting Large Language Models as Zero-shot Relation Extractors (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies show that large language models (LLMs) transfer well to new tasks out-of-the-box . relationship extraction (RE) involves a certain degree of labeled or unlabeled data even under zero-shot setting. |
| Approach: | They propose a simple prompt recursively using LLMs to transform RE inputs to QA format . they propose qq prompting and qt prompting to improve their results . |
| Outcome: | The proposed method improves on different model sizes, benchmarks and settings. |
Soft Knowledge Prompt: Help External Knowledge Become a Better Teacher to Instruct LLM in Knowledge-based VQA (2024.acl-long)
Copied to clipboard
| Challenge: | Recent research focuses on improving prediction performance and reliability of LLM. |
| Approach: | They propose a method to actively extract valuable information from the knowledge to produce a latent vector as a soft prompt, which is fused with the image embedding to form a knowledge-enhanced context to instruct LLM. |
| Outcome: | The proposed method improves performance on knowledge-based VQA benchmarks. |
A Table-to-Text Framework with Heterogeneous Multidominance Attention and Self-Evaluated Multi-Pass Deliberation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Table-to-text works have been widely applied in different domains, such as weather forecast and financial report generation. |
| Approach: | They propose a table-to-text approach on top of Self-evaluated multi-pass Generation and Heterogenous Multidominance Attention to explore the hierarchical structure. |
| Outcome: | The proposed method outperforms several SOTA methods quantitatively and qualitatively on three public datasets. |
Are You Copying My Model? Protecting the Copyright of Large Language Models for EaaS via Backdoor Watermark (2023.acl-long)
Copied to clipboard
Wenjun Peng, Jingwei Yi, Fangzhao Wu, Shangxi Wu, Bin Bin Zhu, Lingjuan Lyu, Binxing Jiao, Tong Xu, Guangzhong Sun, Xing Xie
| Challenge: | Large language models (LLMs) have demonstrated exceptional abilities in both text understanding and generation. |
| Approach: | They propose an Embedding Watermark method that implants backdoors on embeddings to protect copyright of large language models. |
| Outcome: | The proposed method protects the copyright of large language models without compromising service quality while minimizing the adverse impact on the original embeddings’ utility. |
DS-MHP: Improving Chain-of-Thought through Dynamic Subgraph-Guided Multi-Hop Path (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing knowledge graph methods lack adaptability in knowledge-intensive tasks with multiple entities and implicit multi-hop relations. |
| Approach: | They propose a zero-shot framework to enhance LLM reasoning in multi-entity relation tasks. |
| Outcome: | DS-MHP outperforms baselines and state-of-the-art methods on 12 datasets spanning commonsense, logical, symbolic, and arithmetic reasoning. |
Contrastive Pre-training for Personalized Expert Finding (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to expert finding are effective for a community question answering platform. |
| Approach: | They propose a CQA-domain Contrastive Pre-training framework for Expert Finding which could learn more comprehensive question representations. |
| Outcome: | The proposed framework could learn more comprehensive question representations on six real-world datasets. |
Capability Decomposition for Unified Information Extraction via Hierarchical Mixture-of-Experts (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for IE tasks suffer from inconsistent schema representation and implicitly intermediate reasoning . UC-UIE adopts a low-rank adapted hierarchical Mixture-of-Experts adapter for UIE tasks . |
| Approach: | They propose a framework that decomposes IE reasoning into three universal capabilities . UC-UIE adopts a low-rank Adaptation adapter to fine-tune LLMs for IE tasks . |
| Outcome: | The proposed framework outperforms full-parameter tuning methods with 1.24% trainable parameters and outperformed existing methods in generalization and interpretability. |
LLM-Guided Semantic-Aware Clustering for Topic Modeling (2025.acl-long)
Copied to clipboard
| Challenge: | Experimental results show that topic modeling is competitive compared to closed-source methods. |
| Approach: | They propose a semi-supervised topic modeling method that combines LLMs with clustering to improve topic generation and distribution. |
| Outcome: | The proposed method outperforms state-of-the-art methods that utilize GPT-4 on topic alignment and exhibits competitive performance compared to Neural Topic Models on topic quality. |
Boosting Textural NER with Synthetic Image and Instructive Alignment (2024.findings-acl)
Copied to clipboard
| Challenge: | Named entity recognition (NER) is a key task reliant on textual data. |
| Approach: | They propose a method to transform NER into a multimodal task by using images from the internet as auxiliaries. |
| Outcome: | The proposed method surpasses all text-only baselines and improves F1 score by 1.4% to 2.3% on prominent MNER datasets. |
Unlocking Instructive In-Context Learning with Tabular Prompting for Relational Triple Extraction (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for relational triple extraction (RTE) are unnatural and recast RTE tasks to text-to-text prompting formats. |
| Approach: | They propose a tabular prompting for RTE which frames RTE task into a table generation task and propose an instructive in-context learning which only selects and annotates samples considering triple semantics in massive unlabeled samples. |
| Outcome: | The proposed prompting for RTE with TableIE achieves state-of-the-art performance compared to other methods . the proposed prompts are based on off-the shelf LLMs and are scalable to multiple scenarios . |
Exploring Layer Activation Dynamic of CoT via Knowledge Probe (2026.acl-long)
Copied to clipboard
| Challenge: | Chain-of-thought reasoning has emerged as a crucial paradigm for multi-step reasoning tasks. |
| Approach: | They propose a multi-stage probing framework that enforces structured reasoning with three explicit stages: keyword extraction, theorem generation, and computation execution. |
| Outcome: | The proposed framework enforces structured reasoning with three explicit stages: keyword extraction, theorem generation, and computation execution. |