Papers by Ashton Anderson
SPIN: Sparsifying and Integrating Internal Neurons in Large Language Models for Text Classification (2024.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have revolutionized text classification, but current paradigms rely on output of final layer . implicit internal structures that contribute to LLMs' impressive performance are neglected, forgoing potential performance gains. |
| Approach: | They propose a model-agnostic framework that sparsifies internal neurons of intermediate layers of LLMs for text classification. |
| Outcome: | The proposed framework significantly improves text classification accuracy, efficiency and interpretability. |
ChatBench: From Static Benchmarks to Human-AI Evaluation (2025.acl-long)
Copied to clipboard
| Challenge: | In 2024, 40% of US adults reported using generative AI in their everyday lives, an unprecedented rate of adoption for a new technology. |
| Approach: | They propose to convert MMLU questions into user-AI conversations by seeding the user with the question and having them carry out a conversation with the LLM to answer their question. |
| Outcome: | The proposed model can estimate user-AI accuracy by fine-tuning a user simulator on a subset of ChatBench. |
Efficient Transformer Knowledge Distillation: A Performance Review (2023.emnlp-industry)
Copied to clipboard
| Challenge: | Pretrained transformer language models have been gaining popularity in the field of natural language processing . however, there is no study into the intersection of these two fields . |
| Approach: | They propose a method to extract knowledge from transformers to produce high-performing efficient attention models with low costs. |
| Outcome: | The proposed model compression method preserves up to 98.6% of original model performance across short-context tasks and up to 95.8% on long-concept Named Entity Recognition tasks while decreasing inference times by up to 57%. |
Tandem Training for Language Models (2026.eacl-long)
Copied to clipboard
| Challenge: | As language models improve, their actions and reasoning will become difficult or impossible for weaker agents and humans to follow, undermining interpretability and oversight. |
| Approach: | They propose a tandem training paradigm that allows models to adapt their language to weaker partners by intermittently and randomly sampling a frozen weak model instead of the strong model being trained. |
| Outcome: | The proposed model is able to produce solutions that are intelligible to weaker agents and humans while keeping task accuracy high. |
LLM Safety From Within: Detecting Harmful Content with Internal Representations (2026.acl-long)
Copied to clipboard
| Challenge: | State-of-the-art guard models rely on terminal-layer representations and overlook safety-relevant features encoded across internal layers. |
| Approach: | They propose a lightweight guard model that harnesses safety neurons from LLM internals without modifying the underlying model. |
| Outcome: | The proposed model outperforms open-source guard models across multiple benchmarks while using 250 fewer trainable parameters. |