Papers by Lei Hsiung
HTMuon: Improving Muon via Heavy-Tailed Spectral Correction (2026.findings-acl)
Copied to clipboard
| Challenge: | Muon’s orthogonalized update rule suppresses the emergence of heavy-tailed weight spectra and over-emphasizes the training along noise-dominated directions. |
| Approach: | They propose to preserve Muon's ability to capture parameter interdependencies while producing heavier-tailed updates and inducing heavier-tail weight spectra. |
| Outcome: | The proposed algorithm suppresses the emergence of heavy-tailed weight spectra and over-emphasizes training along noise-dominated directions. |
Spectral Insights into Data-Oblivious Critical Layers in Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have identified critical layers linked to specific functions or behaviors, limiting their use to post-hoc settings. |
| Approach: | They propose a data-oblivious approach to identify intrinsic critical layers in pre-fine-tuned LLMs by analyzing representation dynamics via Centered Kernel Alignment. |
| Outcome: | The proposed approach identifies critical layers in pre-fine-tuned models . layers with significant shifts in representation space are also those most affected during fine-tuning . |
Why LLM Safety Guardrails Collapse After Fine-tuning: A Similarity Analysis Between Alignment and Fine-tuning Datasets (2026.acl-long)
Copied to clipboard
| Challenge: | Existing mitigation strategies focus on reactively addressing jailbreak incidents after safety guardrails have been compromised. |
| Approach: | They investigate the degradation of safety guardrails through the lens of representation similarity between upstream alignment datasets and downstream fine-tuning tasks. |
| Outcome: | The proposed model reduces harmfulness score by 10.33% when compared to baseline models. |