Papers by Hairu Wang
Pruning via Merging: Compressing LLMs via Manifold Alignment Based Layer Merging (2024.emnlp-main)
Copied to clipboard
Deyuan Liu, Zhanyue Qin, Hairu Wang, Zhao Yang, Zecheng Wang, Fangying Rong, Qingbin Liu, Yanchao Hao, Bo Li, Xi Chen, Cunhang Fan, Zhao Lv, Dianhui Chu, Zhiying Tu, Dianbo Sui
| Challenge: | Existing methods for parameter pruning fail to utilize the knowledge from pruned parameters. |
| Approach: | They propose a method that uses manifold learning and the Information Bottleneck measure to merge similar layers to preserve model performance. |
| Outcome: | The proposed method outperforms pruning methods on multiple datasets and LLMs with quantization and achieves substantial compression ratios. |
FRAG: A Flexible Modular Framework for Retrieval-Augmented Generation based on Knowledge Graphs (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for retrieval-augmented generation struggle with a trade-off between flexibility and retrieval quality. |
| Approach: | They propose a flexible modular KG-RAG framework that uses query text instead of KGs . they propose to use query text to infer the structural information of reasoning paths . |
| Outcome: | The proposed method achieves state-of-the-art performance with high efficiency and low resource consumption. |
SkewRoute: Training-Free LLM Routing for Knowledge Graph Retrieval-Augmented Generation via Score Skewness of Retrieved Context (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models incur high inference costs during deployment, causing hallucination . no dedicated routing methods exist for RAG, and existing training-based routers face challenges scaling to this domain . |
| Approach: | They propose a plug-and-play routing framework that optimizes performance and cost . the framework delivers over 3x higher routing effectiveness while reducing runtime to less than 0.001x . |
| Outcome: | The proposed framework delivers over 3x higher routing effectiveness while reducing runtime to less than 0.001x compared to existing methods. |