Papers by Truyen Tran
Do Domain-specific Experts exist in MoE-based LLMs? (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on domain-specific experts in Large Language Models (LLMs) are still lacking. |
| Approach: | They propose a training-free framework that introduces zero additional inference cost and outperforms well-trained MoE-based LLMs. |
| Outcome: | The proposed framework outperforms well-trained MoE-based LLMs and strong baselines across target and non-target domains. |
Continual Safety Alignment via Gradient-Based Sample Selection (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models require continuous adaptation to new domains, tasks, and evolving requirements. |
| Approach: | They propose a gradient-based sample selection method that filters high-gradient samples during fine-tuning. |
| Outcome: | The proposed method significantly improves alignment preservation while maintaining competitive task performance on continual domain tasks. |
SimSMoE: Toward Efficient Training Mixture of Experts via Solving Representational Collapse (2025.findings-naacl)
Copied to clipboard
| Challenge: | Sparse mixture of experts (SMoE) is an effective approach for scaling large language models with a constant computational cost. |
| Approach: | They propose a similarity-based sparse mixture of experts (SimSMoE) that guarantees a solution to address the representation collapse problem between experts given a fixed FLOPs budget. |
| Outcome: | The proposed method outperforms existing methods and outperformed other SMoE routing methods in performance. |