Papers by Dongzhou Cheng
Dropping Experts, Recombining Neurons: Retraining-Free Pruning for Sparse Mixture-of-Experts LLMs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Sparse Mixture-of-Experts (SMoE) architectures require loading all expert parameters . previous work focused on expert pruning and merging but focused on neuron-level structure . |
| Approach: | They propose a task-agnostic framework for expert pruning and reconstruction . it prunes redundant experts using router statistics, then decomposes them into neuron-level expert segments . |
| Outcome: | The proposed framework reduces the number of experts and memory usage, making it easier to deploy. |
One Refiner to Unlock Them All: Inference-Time Reasoning Elicitation via Reinforcement Query Refinement (2026.acl-long)
Copied to clipboard
| Challenge: | Existing alignment methods for Large Language Models (LLMs) are expensive and lack the flexibility to fully activate their latent reasoning capabilities. |
| Approach: | They propose a modular framework that treats reasoning elicitation as an inference-time alignment task. |
| Outcome: | The proposed framework outperforms baselines by 2.1% on average across diverse architectures and benchmarks. |