Papers by Difan Jiao
SPIN: Sparsifying and Integrating Internal Neurons in Large Language Models for Text Classification (2024.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have revolutionized text classification, but current paradigms rely on output of final layer . implicit internal structures that contribute to LLMs' impressive performance are neglected, forgoing potential performance gains. |
| Approach: | They propose a model-agnostic framework that sparsifies internal neurons of intermediate layers of LLMs for text classification. |
| Outcome: | The proposed framework significantly improves text classification accuracy, efficiency and interpretability. |
Deciphering Cultural Representations in Large Language Models via Sparse Autoencoders (2026.findings-acl)
Copied to clipboard
| Challenge: | Prior work has identified so-called cultural neurons, but individual neurons are often polysemous, conflating abstract cultural knowledge with surface-level lexical cues due to superposition. |
| Approach: | They apply Sparse Autoencoders to decompose LLM activations into sparse, interpretable feature representations that disentangle culturally selective features. |
| Outcome: | The proposed model disentangles culturally selective features from paraphrasing and task formats, indicating abstraction beyond lexical correlations. |
LLM Safety From Within: Detecting Harmful Content with Internal Representations (2026.acl-long)
Copied to clipboard
| Challenge: | State-of-the-art guard models rely on terminal-layer representations and overlook safety-relevant features encoded across internal layers. |
| Approach: | They propose a lightweight guard model that harnesses safety neurons from LLM internals without modifying the underlying model. |
| Outcome: | The proposed model outperforms open-source guard models across multiple benchmarks while using 250 fewer trainable parameters. |