Papers by Pengpeng Shao
Two-Stage Regularization-Based Structured Pruning for LLMs (2026.acl-long)
Copied to clipboard
Mingkuan Feng, Jinyang Wu, Siyuan Liu, Shuai Zhang, Hongjian Fang, Ruihan Jin, Feihu Che, Pengpeng Shao, Zhengqi Wen, Jianhua Tao
| Challenge: | Structural pruning is a promising solution for large language models . prior structured pruning methods remove unimportant parameters based on certain metrics . |
| Approach: | They propose a structural pruning method that iteratively learns the weights of transformer layers by adding their l1-norm to the loss function. |
| Outcome: | The proposed pruning method outperforms strong layer-wise pruning methods without requiring retraining. |
RadialRouter: Structured Representation for Efficient and Robust Large Language Models Routing (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Current routing methods are limited in exploring the connection between query and LLM characteristics. |
| Approach: | They propose a framework for LLM routing that uses a transformer-based backbone and a radial structure to articulate the query-LLMs relationship. |
| Outcome: | The proposed framework outperforms existing routing methods by 9.2% and 5.8% on RouterBench. |
Pandora’s Box or Aladdin’s Lamp: A Comprehensive Analysis Revealing the Role of RAG Noise in Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Retrieval-Augmented Generation (RAG) has emerged as a promising approach to address hallucinations in large language models (LLMs). |
| Approach: | They define seven distinct noise types from a linguistic perspective and establish a Noise RAG Benchmark (NoiserBench) they propose to evaluate noise that is beneficial to LLMs and noise that's harmful to LRMs. |
| Outcome: | The proposed framework consists of seven distinct noise types from a linguistic perspective and includes multiple datasets and reasoning tasks. |